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Contracts  and  Contractors 


Foreword 


Rc.v  Admiral  Rowland  <>'  f reeman  III 


Past  Performance:  An  Essential  Element  in  Source  Selection 

Colonel  .Michael  -t  \asar 
It  has  lon/t  been  reisigniscii  that  a contractor's  " track  res'orxl"  should  he 
one  of  the  /actors  inffuencnig  v<  mice  seln'lion.  however.  /X)/)  has  not  yet 
deusnl  an  effeitne  means  of  utilizing  /si si  performance  data  In  this  article. 
Colonel  Xassr  prop  >ses  a methixl  hy  which  a contractor'',  past  perfomiame 
can  he  accurate ly  eialuansl  and  efficient! ’»  applies!  hi  source  selection 
actn  dies 

Toward  More  Effective  Implementation  of  Specification 
Tailoring  Dr  Warren  I Mathews 

■t Dei  idem itym/!  the  principal  participants  m the  tail. amt:  prtvess  and 
assessm/t  the  key  influences  < in  them.  Dr  Mathew » offers  m'ommendalions as 
to  hint  these  participants  might  he  motnatesl  tow  an!  more  effivlne  imple- 
men  tat  n m i if  the  taili  irm/t  ct  incept 

Surveillance  of  Defense  Programs:  The  Industry  Role 

lr\ in/;  .1  Sandlei 

Determining  the  app.  ,ate  degree  < 'fgoi  eminent  sun  eillance  < if  defense 
contractois  is  of  continuing  concern  to  the  go\ eminent  and  the  contractor 
alike  In  this  article.  Mr  Sandler  talks  ahmil  the pmhlem  fn>ni  the  i lewp'int 
i'f  the  Defense  Contract  Audit  Agency  and  suggests  actions  which  the 
contractor  may  rake  to  ns/ttce  the  amount  of  goi  eminent  mnei/lance 
n\/uiml 

Monitoring  the  Government/Industry  "Partnership” 

/ icu tenant  t < >/.  mel  M K Mi  'ntgi  'men 
I'he  "partnership" between ginemnieiit  and mdustn  that  is  initiates!  with 
the  signing  of  a defense  s\>ntract  is  ,ii  • that  must  he  constant Is  monitoml  if 
the  contract  is  to  he  snnsithly  executes!  lieutenant  t'o/onel  Montgomery 
offers  adi  we  for  those  charges!  u nh  momtonrig  that  partnership 
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The  Two-Tier  Matrix  Organization  in  Projec  t Management 

Or  William  C Wall,  Jr 
the  use  i <!  I he  ir.-mx  form  of  organisation  in  project  management  is 
oi'mmon.  bill  nnist  program  I'tlh'es  hair  been  organises!  along  the  lines  of  a 
single-tier  matin  In  this  article.  Or  Wall  outlines  the  two-ner  organisation 
anil  cites  its  ail*  .images  f >i  pn  'lis  t management 
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Management  Update:  The  Army  Science  and  Technology 
Program  Or  Manm  E I asset 

flexibility  is  a key  uord  in  management  h day  Or  l asser  outlines  the 
meihnl  hi  nhich  Army  labswati'n  Jim-tors  haie  been  gtien  substantial 
flexibility  in  managing  the  Army  \ science  ami  technoli'gy  program 


54 

66 

74 


Software  Quality  and  Productivity  H M Knight 

I his  article  emphasises  the  nccsl  for  applying  the  quality  assuram'c 
discipline  to  stilhxare  and  outlines  methsds  tor  mluctng  soil u are  failures  bi 
eliminating  the  ern  us , n delis  ts  that  cause  them 

New  Direc  tions  for  NATO  Or  William  J ferry 

In  this  article,  bases!  on  remarks  be  fur  the  Axialion  HW't  and  Spice 
I'es'hnology  ('onterence  at  Hrussels.  Helgium.  Oi  I’erry  ticuses  on  the 
challenges  lacing  the  V I K > 4 l/iaris'c  and  efforts  currently  underway  to  meet 
thi  "c’  challenges 
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Winter  I'W  Technology  Transfer 
Spring  Joint  Program  Management 


FOREWORD 


HRc.tr  Admirxl  Rowland  O.  Freeman  III 

Commandant. 
Defense  Systems  Management  College 

Much  of  this  issue  of  the  Renew  is  focused  on  the  contract  and  the  contractor, 
which  makes  it  appropriate  that  1 say  a few  words  about  the  contracting  phase  of  the 
acquisition  process.  It  has  not  been  uncommon  for  the  acquisition  manager  to  blame 
the  contracting  effort  for  failures  in  the  acquisition  process  Various  types  of 
contracts  have  been  w ritten  for  the  several  phases  of  the  acquisition  process — cost- 
plus-fued-fec  contracts  for  production;  total  package  procurement;  fixed-price 
research  and  development  contracts;  incentive  contracts  for  research,  development 
and  production,  redetermitiablc-downward-only  contracts;  letter  contracts;  and  on 
and  on.  each  one  intended  to  prevent  acquisition  shortcomings.  The  list  of  ways  we 
have  tried  to  use  the  contracting  process  to  overcome  a lack  of  good  specifications. 
piH'r  definition  of  tasks,  oxer  optimistic  schedules,  and  poor  program  estimation  is  a 
lengthy  one.  However,  at  the  Hershcy  conference  on  acquisition  research  held  31 
May-2  June  I^TS.  there  was  general  agreement  that  we  must  examine  the  basics  of 
our  acquisition  process  and  make  changes  there  rather  than  attempt  to  correct 
deficiencies  in  the  process  through  the  use  of  contractual  gimmicks. 

Farly  communication  between  managers  and  technical  and  contracting  person- 
nel must  be  established  to  promote  understanding  by  all  parties.  The  spectrum  of 
actions  required  for  successful  contracting  necessitates  a sound  understanding  and 
amalgamation  not  only  of  the  technical  requirements  but  also  of  the  business  and 
contractual  actions  required  to  consummate  a good  contract  A sound  contract,  or 
the  "right"  t>  pe  of  contract,  is  the  instrument  to  promote  understanding  between  the 
buver  (government!  and  the  seller  (industry!,  but  it  cannot  compensate  for  inade- 
quate planning  and  execution  by  the  entire  government  team 

If  the  contracting  phase  of  the  acquisition  process  is  in  need  of  more  research  and 
study,  it  is  only  one  of  many  aspects  of  acquisition  management  that  deserve 
attention  Some  examples  of  major  areas  of  research  needing  emphasis  arc  phase  zero 
activities  in  the  acquisition  policy  cycle,  tools  for  meaningful  delineation  of  the  range 
of  life  cycle  costs,  quantitative  measurement  techniques  for  acquisition  management 
trade  offs,  competition  in  the  acquisition  cycle,  the  relationship  of  the  Program 
Planning  budget  System  and  the  Defense  Acquisition  Review  Council  process; 
commercial  standards  versus  government  specifications,  and  the  impact  of  NATO 
rationalization,  standardization  and  interoperability  on  the  acquisition  process;  to 
mention  only  a few  DSMC  will  chair  with  the  Federal  Acquisition  Institute  the  next 
Acquisition  Research  Symposium,  to  be  held  at  the  Naval  War  College  in  May  |U7U 
We  are  looking  forward  to  addressing  many  of  the  above  issues,  as  well  as  those 
which  may  be  contributed  by  readers  of  the  Wei  iew 

Only  through  sound  research  can  we  develop  meaningful  and  useful  acquisition 
policy  for  the  future  l et  us  hear  from  you 
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M ith  this  issue,  the  IVt'ense  Systems  Management  Rex  lew  begins  publication  in  a 
totally  net*  tomtit  M e rec\gnire  that  a .'hinge  c V'this  magnitude  carries  with  it  in 
element  it fnsk.  in*./  it  his  mv  Hith.mr  a great  Jew/  cV'fivethcHighr  that  the  changes 
»ere  undertaken  It  is  iHir  feeling,  thinner,  that  this  tlvmat  change  was  the  neat 
logical step  in  the c witinuing  eio/uthm  > V'the Defense  Systems  Management  Rex  text 
Frxmi  its  inception.  the  Rex  texx  has  been  kxiked  up.ni  as  a professional jcvimal — a 
torum  Sir  the  exchange  .V  ideas  in  defense  systems  management  H'hile  from  all 
acciHints  the  Rex  texx  has  ic'hteied  success  in  that  regard,  we  are  constantly  stmirtg 
re  do  the  gib  better  the  changes  reflected  in  this  issue  represent  the  latest  but 
certainly  net  rhe  last  cV'cHir  efforts  in  that  direction  \<v  has  cHitnard  appearance 
been  the  se/e  fivus  of' our  attentien  Slot  mi  ths  finding  the  Met  uhan  apwhegm  that 
"T he  medium  is  the  message. " a pub/rcathw  nses  »v  tails  en  the  strength  .it'  its 
ccmtenrs  her  that  reasim.  we  are  continuing  research  tie  and  publish  papers  that  we 
think  m v .inly  add  te  the  bdy  cV'kn.iw  ledge  in  defense  acquisition  management,  but 
alsc'  pn'vide  intomiation  applicable  in  the  Jay-ro^iiy  acfinties  of' the  pmV'essienal 
acquisition  manager  Ibis  is  iR ir  nnist  difficult,  it'  t\ir  most  impyirtant.  task.  As 
alwayx  we  muteywir omimenrs  and cntK'tsms 


l'he  Fditonal  Staff 
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PAST  PERFORMANCE: 

AN  ESSENTIAL  ELEMENT  IN 
SOURCE  SELECTION 

Colonel  Michael  A Xassr 


The  concept  of  rewarding  or  penalizing  defense  contractors  based  on  their  past 
performance  is  certainly  not  new  Hitch  and  McKean  identified  its  importance  in 


l»hO: 


Nothing  spurs  a contractor  as  effectively  as  knowledge  that  his  perfor- 
mance will  be  compared  directly  with  that  of  a rival  or  rivals,  with 
appropriate  rewards  and  penalties — either  in  the  short  run  (by  the  terms  of 
the  current  contract)  or  the  somewhat  longer  run  (in  the  next  or  later 
contracts).1 

Frederick  M Scherer  thoroughly  explorer!  the  concept  of  competition  based  on 
contractor  reputation  and  identified  numerous  difficulties  with  such  a system, 
including  the  problem  of  time,  quality,  and  cost  weighting,  the  difficulty  of 
measurement  without  the  influence  of  biases;  and  the  problem  of  blending  givs.1  and 
poor  jobs  together  to  obtain  an  overall  index  of  contractor  performance1 

Today's  Defense  Acquisition  Regulation  (DAR)  states  that  "...  the  Contracting 
Officer  shall  consider  not  only  technical  competence,  but  also  all  other  pertinent 
factors  including  management  capabilities,  cost  controls,  and  past  performance  tn 
adhering  to  contract  requirements,  weighing  each  factor  in  accordance  with  the 
requirements  of  t he  particular  procurement 

The  Air  Force  regulation  on  source  selection  outlines  a separate  section  on 
"Offerors'  Past  Performance"  as  part  of  the  Source  Selection  Advisory  Council 
Analysis  Report.*  and  the  former  weighted  guidelines  method  of  the  Armed  Services 
Procurement  Regulation  (ASPR)  included  a factor  for  contractors’  past  perfor- 
mance in  computing  the  profit  or  fee  object i\  e.' 


V J Hitch  jhuI  R N McKeon.  KwNvmoixV'/ \rfrtis? inth**  \iK'fo*i  AXt*(Ssint»  Monte*,  Calif  The 
Rami  Corporation.  Marvh  1*>C»0V  pp  ' '2  2.V* 

’Frederick  M Scherer.  I'ht'  W<\*/\v»*  A*\jinsin%*t  K wnvtik*  lr\xvin*e>  ( IW  //>  (Boston 

Mars  *ut  Bujunrw  School.  Di\ rston  of  Research.  l'JMV  pp  101 

‘IVpartment  oflVfense.  \mK\1  Srrwx*  ftw'urmtrtM  Rr$ukt\vi,  Edition  (Washington.  O C 
I’  S Ooifmnwit  Pnnting Office  V pp  l 'AV*  1 (n0 

\A»r  Force  Regulation  ‘’IV  IV  .SymrSirAvravr  MVy*  mi\1  f'rwwUnrx  In  .Apnl  l^d.  p A 4 

*F  ka\afav  t'*v un*r%\  /Vw  /VrformarKr  Aww  fiv  i',vmaK7  4m anL  Professional  Study  No  t*»40 
i Ma\*  ell  AFH.  AU  Ah  Wai  College,  Apnl  W*0V  p '' 
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Fivtr  Swtrm*  c FAtvmviv  Swtems  />»  »xav i /Vr»  n *av  *v irK  hnie « tunr*  Jtv  / Vrvr»v 

*V  \1jinuf'j*  (unnji.  H?v*Kju*riirr\  Air  Mvtr  >'» />m7iv  .V  Vwomx  h'r^nx^nnj: 
4<*nvi4urvji/  >'» strmx  />iwtv?.  and  /Vtyntm  \t.in\cn  m th?  An  ljutKh<\i  iYui«r  W iwik* 
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IVspite  (his  apparent  recognition  of  the  importance  of  a contractor's  "track 
record."  the  Department  of  Defense  (DOD)  currently  displays  scant  evidence  of 
placing  meaningful  emphasis  in  this  area.  It  is  not  uncommon  to  find  a case  in  which 
a company  receives  a very  desirable  new  program  award  even  though  that  compa- 
ny's performance  on  an  earlier  contract  was  essentially  unsatisfactory. 

This  situation  is  in  stark  contrast  to  that  .if  the  private  sector,  where  organiza- 
tions generally  maintain  a list  of  preferred  vendors  and  suppliers.  The  "preferred 
list"  is  primarily  determined  by  experience  from  previous  contracts  and  frequently  is 
the  major  influence  in  the  determination  of  future  awards  * 

Competition  in  the  defense  marketplace  is  such  that  contractors  give  top  priority 
to  moving  into  promising  new  fields,  thereby  promoting  capabilities  for  winning 
future  programs.  Achieving  good  performance  on  current  programs  becomes  of 
secondary  importance.  Unfortunately,  there  is  no  well-defined  program  in  the 
Department  of  Defense  to  effectively  counter  this  situation. 

Thus,  today’s  source  selection  authority  (SSA)  is  missing  an  important  and 
powerful  tool  for  making  the  selection  decision.  Although  he  may  be  presented  with 
some  subjective  and  sketchy  information  on  past  performance,  it  is  generally  lacking 
in  depth  and  consistency. 

Lessons  of  l he  Past 

What  can  be  done  to  rescue  the  source  selection  authority  from  this  dearth  of 
meaningful  past  performance  data?  Since  the  problem  is  not  a new  one,  there  is 
ample  information  from  the  past  which  bears  examination 

In  the  l^bOs  the  Department  of  IVfense  developed  considerable  experience  with 
its  Contractor  Performance  Evaluation  (CPF I system.  The  system  encompassed 
RAD  of  $2  million  per  year  or  $10  million  overall,  and  production  programs  of  $10 
million  per  year  or  $20  million  overall  ' The  CPE  system  required  that  a total  of  eight 
DOD  forms  be  completes!  on  a semiannual  basis  by  project  managers,  service 
evaluation  groups,  and  contractors.’ 

There  was  an  extensive  submission  and  review  process  for  these  forms  (Figure  D 
pnor  to  their  being  forwarded  to  the  DOD  Data  Bank  at  the  IVfense  Documenta- 
tion Center  for  future  use  by  source  selection  organizations,  contracting  officers  and 
the  renegotiation  board.’ 


*J  Ronakl  Vox.  Arming  Xmerva  ( Boston  Graduate  School  of  Business  Administration.  Harvard 
University.  I *74kpp  XU  '04 

'Ah  J-otve  Institute  of  Tei-hnotogy.  SrtnieJ  Headings  Hns-unmient  Kef'nrsher  Cisirse  tWnshl 
Pal  ter.  on  A HI,  Ohio  Seh.sO  of  Systems  and  l ojisties.  10'01.  pp  VI-4 
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Figure  1 

FORMER  OOD  SYSTEM  FOR  EVALUATION  OF  A CONTRACTOR  S 
PERFORMANCE 
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When  it  was  initiated  in  I'FftT.  the  CPF  program  was  designed  to  he  fact -oriented 
to  the  maximum  extent  possible  There  were  provisions,  however,  for  subjective 
comments  by  both  the  project  manager  and  the  contractor.  This  aspect  of  the 
program  allowed  it  to  degenerate  into  a senes  of  verbal  duels  between  project  offices 
and  contractors  whenever  honest  differences  of  opinion  arose." 

A survey  conducted  in  I*>h7  by  the  Air  Force  Systems  Command  (AFSO  found 
that  CPF  data  was  being  used  in  varying  degrees  by  source  selection  advisory 
councils,  but  there  was  only  one  instance  where  it  proved  to  be  a decisive  factor  in 
contract  award  " Thus,  there  was  no  firm  evidence  that  the  program  was  achieving 
its  prime  objective  despite  years  of  operation  and  extensive  involvement  of  Kith 
government  and  contractor  personnel. 

In  November  of  WO  the  program  was  formally  cancelled  for  being  neither  cost 
effective  nor  useful  for  source  selection.  Although  the  sophistication  and  the  volume 
of  paper  that  it  generated  helped  lead  to  its  demise,  a prune  cause  of  the  program's 
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termination  was  its  failure  to  employ  a methodology  that  would  allow  source 
selection  authorities  and  contracting  officers  to  consistently  apply  the  available  data. 

Experience  from  the  CPE  of  the  1960s  points  to  the  fact  that  a successful 
program  of  the  future  must  emphasize  simplicity  and  the  prime  elements  of  cost, 
schedule  and  technical  performance.  Subjective  narrative  assessments  should  be 
avoided  and  a methodology  for  utilization  must  be  developed,  enforced,  and 
continuously  tracked  to  determine  its  effectiveness. 

Proposed  Contractor  Performance  Program 

There  is  still  a need  for  a structured  program  to  measure  and  take  into  account  a 
contractor’s  past  performance.  Few  dispute  that  past  performance  should  be 
considered  somewhere  in  the  source  selection  process.  Diverse  opinions  exist, 
however,  as  to  the  type  of  information  required  and  the  manner  in  which  it  should  be 
employed. 

Proposed  herein  is  a simplified,  fact-oriented  data  system  that  records  the  cost, 
schedule  and  performance  status  of  major  Department  of  Defense  contracts  that 
have  been  active  in  recent  years.  Included  is  a suggestion  on  how  this  information 
can  be  used  by  source  selection  activities.11 

Initially  it  is  recommended  that  the  information  be  accumulated  on  Department 
of  Defense  contracts  over  $5  million  which  are  currently  active  or  which  have  been 
completed  during  the  past  three  years.  The  data  generated  would  be  concise  and 
broken  into  four  areas:  (1)  administrative,  (2)  cost,  (3)  schedule,  and  (4) 
performance: 

• Administrative.  A listing  of  the  contract  number;  dollar  value;  procuring  agency; 
acquisition  phase  (advanced  development,  production,  etc.);  a brief  description 
of  work;  names  and  telephone  numbers  of  the  government  project  manager, 
procuring  contracting  officer,  and  administrative  contracting  officer;  dates  of 
contract;  and  type  of  contract. 

• Cost.  Percent  over  or  under  target  and  dollar  amount  (actual  for  completed,  and 
estimated  at  completion  for  active  contracts);  number  and  dollar  amounts  of 
claims  submitted  and  claims  approved/disapproved. 

• Schedule.  Months  the  contract  has  been  delinquent/total  contract  months; 
reasons  for  delinquencies;  changes  made  to  original  schedule  and  reasons  for 
them. 

• Performance.  Number  of  DD  250  (Material  Inspection  and  Receiving  Report) 
acceptances;  numbers  of  major  deficiencies  and  conditional  acceptances  on  DD 
250s;  numbers  of  deviations  and  waivers;  numbers  of  specifications  and  test 
plans/reports  resubmitted  for  approval.  (Figure  2) 

"M.  L.  Fowler,  Assistant  Deputy  for  Procurement,  Electronic  Systems  Division.  Air  Force  Systems 
Command,  designed  the  original  data  base  from  which  the  proposed  contractor  past  performance  system 
has  been  developed 
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Figure  2 


ADMINISTRATIVE 

Contract  Number: 

Dollar  Value: 

Procuring  Agency: 

Acquisition  Phase:  (Development  or  Production) 

Brief  Desc  ription  of  Work: 

Name  and  Telephone  No.  of:  Government  Program /Project  Manager: 

PCO: 

ACO: 


Dates  of  Contract: 

Type  of  Contract  (e  g.  FTP,  C'PIF . etc  .): 
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l'h«*  pro|x>sed  information  can  he  gathered  in  a numbei  of  ways  Ihitil  a DOD 
wule  system  or  leqiiirrment  is  established.  requests  loi  proposals  may  be  used  to 
solicit  the  desired  int'ormatiou  from  olTerors  This  data  couhl  then  he  validated  in 
pre-award  surveys  lhe  mere  lad  that  such  information  is  being  requested  from 
oiTerors  will  serve  to  pul  them  on  notice  that  the  Department  of  Defense  is  giving 
added  emphasis  to  contractors’  past  performance  This  type  of  information  could 
also  he  mamtamed  by  DOD  administrative  contracting  officers,  w ho  would  update  it 
as  requited  in  ordei  to  submit  it  to  procuring  agencies  in  conjunction  with  contractoi 
proposals  It  is  ini|vortant  that  the  contractor's  most  recent  ev|verience  be  included, 
as  recently  ruled  by  the  l ieneral  Accounting  Office  (l  i AO). " 

An  Force  Systems  Command  already  has  a computerized  procurement  informa 
lion  program,  the  Autoniatevl  Management  Information  System  (AMIS),  which 
could  possible  be  tapped  to  add  the  above  data  on  contractor  performance  An 
addition  to  the  current  AMIS  could  be  made  so  that  the  information  would  he 
continuously  and  immediately  available  to  all  organizations  m AFSC  Since  the 
Defense  I ogistics  Agency  will  he  exchanging  privuremeiit  information  with  AMIS, 
needed  information  could  be  included  on  all  contracts  administered  by  the  Defense 
Contract  Administration  Service  (IX'AS) 

Vtili/ation  in  Source  Selection 

The  method  and  form  in  which  this  information  is  used  can  be  standardized  to  a 
simple,  basic  report  for  presentation  to  source  selection  authorities  The  cost 
information  would  reflect  the  numbei  of  contracts  which  met  ot  were  over  01  undei 
target  costs  Schedule  information  would  reflect  total  months  of  original  contracts 
and  the  number  of  months  delinquent  and  or  of  schedule  extension  Die  technical 
performance  portion  would  rellcvl  the  quality  of  the  products  delivered  bv  indicating 
the  numbei  ol  dev latious.  conditional  acceptances,  etc  , on  a percentage  basis  relative 
to  the  total  items  delivered  Dollar  values  would  lie  totaled  to  show  gross 
overrun  underrun  for  the  total  contracts  charted  I samples  of  summary  charts 
which  might  he  developed  loi  source  selection  purposes  are  shown  in  Figures  ' and 
4 

Since  standardized  incorporation  of  such  information  m the  source  selection 
piocess  will  constitute  a new  emphasis  on  past  |verformance.  requests  lot  proposals 
must  cleailv  stale  the  maunei  in  which  past  performance  will  he  considered  as  par!  of 
the  criteria  lot  contract  award  Air  Force  Systems  (.'onimand  is  currently  conduct 
mg  a test  in  which  past  performance  is  being  utilized  as  both  a ntaiot  ranked  ot 
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«»'(«•  atra  amt  as  a jicnci itl  rxiiisixlriation  toi  axxant  OtTru'is  \\ ill  tv  uixilrxl  to 
srlcvl  rxmnpln  ol  |vc«>i  |vitoi  manrr  xxtnrh  illustiatr  thr  xx'itliailx'i'x  xa|xahili(x 
irgaixhng  the  x|vmIV  rxaluatix<ii  rnirna  of  the  sxilirilaltxxit 

In  (hr  rxmi  .i  rxintiartoi  has  not  pritormrxl  i»ii  ptrxn'us  IVpaitmrut  »xf  IVlrnsr 
ronliarts,  il  ran  tv  sialrxl  (hat  infounatixxn  irlaiixr  tx>  |vi tot manrr  on  x'oiti|xatahlr 
nx'n  IXMKxxnti arts  max  hr  irxpirsirxl  Inanx  rxrnl,  H \tn>ulxl  nnlhri  hr  imphrst  mn 
mtrmlrxl  (hat  xtala  rx'nlainrxt  in  any  x'ximputriirrxl  xtala  hank  xxill  ronsiiinlr  (hr  stun 
total  of  all  |xast  |vifoimanrr  informal  mu  tx>  tv  xMitsnlrirxt  hx  thr  sxxiuvr  srlrrtmu 
anthxxnty  Othri  prttmrnl  fartual  mfoimatixxii.  snrh  as  that  xxhtrli  might  tv  gathriril 
ftx'in  rontart  xxith  ptx'giatu  inauagris  in  ronliarlmg  prrsxxmtrl,  ran  amt  stiouUI  tv 
inaxlr  axailahlr  loi  tlir  SSA's  rxinstxlriatmn  If  axhhtional  information  is  galhnrxl  x*n 
onr  privsprvtixc  rontiartoi,  thru  roui|xaiahtr  infounation  slmulxl  tv  galhrirxt  i'ii  all 
othris 

Snix'r  thr  pmnr  ol'irrttxr  x>l  ttir  progiam  is  to  tnfhirnrr  sotnw  srlrrtmu,  a 
trrxlhark  sxstrni  is  irspnirxl  tx'  rondnuouslx  rxalnatr  xxhrthri  past  |vitoi  manrr 
intonnatnxn  is  arlnaltx  playing  a ixilr  in  sxxiuvr  srlrx'tion  xlmsmns  A snixrx  loi  in  tx> 
tv  romplrlrxl  t»x  sxuurr  srlrrtmu  authontirs  ami  ot  IHXK  rx'nlil  pnxxnlr  thr 
information  xlrsnrxl 

Summon 

Ihr  pnipxvsr  of  sxxiuvr  srlrx'tion  is  txi  srlrs't  a x'ontraxior,  in>l  inrirlx  tx»  rhxnvsr 
tvtxxrrn  romprimg  pnxpxvsals  Vrt  thr  rmphasis  in  nnvst  tVpaitmrni  (xf  IVlrnsr 
sx'urrr  srlcvtixxiis  is  ox  rtxxhrlmmglx  u|H>n  thr  rxahialixxn  xxf  trrhmral  propxvsals  I hr 
xlilVrrrnrr  tvtxxrrn  rontiartxxis  is  srlxtxxm  sxx  ilhinnnatrxl  as  thr  xhtVrirnrrs  tvtxxrrn 
thru  pmpxvxals 

A prartiral  trpx>ttmg  sxstrni  x»ii  x'xxutiarlxxrs'  |«ast  |vi lx>i manrr  is  m grin  lx  nrrxlrxl 
in  x'lxln  to  rx'nsntri  |xast  as  xxrll  as  pixx|ix'srx<  prrloi manrr  xxlirn  srlrrlmg  IX'O 
ronliartois  A sxstrni  is  prxi|<xvsrxl  xxtnrh  xxill  pi  ox  nit  fartual  xtala  tx'  souivr  srlrrtmu 
groups  xxhilr  not  irspuring  thr  grurratmn  of  x'omphratrxl  x>i  sophist iratrxl  infoinia 
turn  xxtnrh  tiiighi  irx)iitrr  rxprmhtutr  i»f  iiioixhnatr  iiiantnuii's  At  a lain  xlatr  n max 
hr  xlrsiiahlr  txi  xlrxrlx'p  pn  lot  manrr  imhrrs  in  rarh  xif  thr  thirr  airas  xxf  x>vst, 
sx'hrxlulr  ami  priformam-r  With  a pxvsitixr  xlrtriiimialiixu  hx  Ix'p  inauagrmrnt  ami 
xxith  an  x»prn  miml  hx  thxvsr  xxho  must  implrmrnt  sux'h  a pix'giam,  xxr  ran  lakr  a 
giant  strp  tx'xxaitls  improving  thr  rrlatmnship  tvtxxrrn  past  |vi  fx'i  manrr  amt  flitnir 
axxaixls  I 


TOWARD  MORE  EFFECTIVE 
IMPLEMENTATION  OF 
SPECIFICATION  TAILORING 

Dr.  Warren  / Mathews 

One  of  the  more  frequently  discussed  techniques  for  improving  defense  systems 
management  is  "tailoring,"  or  Ihe  selective  application  of  military  specifications  and 
standards  to  an  individual  program  The  purpose  of  this  paper  is  to  present  some 
thoughts  on  what  it  takes  to  make  effective  tailoring  actually  happen  in  the  every  day . 
real  world  The  observations  and  conclusions  presented  here  are  basest  on  a broad 
collection  of  responses  from  senior  representatives  of  the  American  Defense  Pre- 
paredness Association.  the  Aerospace  Industries  Association,  the  Fleclromc  Indus- 
tries Association  and  the  National  Security  Industrial  Association  It  should  be 
made  clear,  however,  that  these  observations  do  not  necessarily  represent  the  official 
positions  of  those  associations. 

1 will  begin  by  identifying  the  principal  participants  in  the  military  specification 
(MU  -Sl’FO  and  military  standard  fMU  -STD)  tailoring  business  and  examining 
f heir  respective  roles  I then  will  assess  the  key  influences  on  those  participants  and 
offer  some  observations  and  recommendations  as  to  how  they  can  be  motivated 
toward  more  effective  implementation  of  tailoring.  Finally,  I will  off'ei  an  oigam/a- 
tional  recommendation  for  facilitating  effective  tailoring  within  the  military  serv  ices 

The  Participants 

I he  principal  participants  in  the  tailoring  business  and  the  communication  and 
authority  relationships  between  them  are  illustrated  in  Figure  1 Because  tailoring  is 
inherently  specific  to  a program,  the  real  center  of  this  activity  is  necessarily  the 
program  office,  including  the  program  manager,  the  contracting  officer,  and  the 
people  who  support  them.  The  other  major  actors  are  the  contractor,  through  w hom 
the  program  office  accomplishes  its  mission;  the  preparing  activities  and  associated 
specialist  communities,  through  which  the  specifications  and  standards  come  into 
existence  and  are  maintained  and  interpreted;  and  the  Department  of  Defense, 
overseer  of  the  entire  enterprise. 

The  program  manager  teceives  direction  through  the  chain  of  command  extend 
mg  down  to  him  from  DOD.  He  in  turn  provides  partial  direction  to  the  contracting 
officer,  who  also  has  a parallel  reporting  path  through  his  functional  segment  of  the 
Civil  Serv  ice  The  specifications  and  standards  of  concern  here  are  generated  and 
maintained  by  preparing  activities,  which  are  identified  by  and  receive  policy- 
direction  from  the  Defense  Materiel  Specifications  and  Standards  Office  (DMSSO) 
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Figure  1 

THE  PARTICIPANTS 


within  the  Office  of  the  Deputy  Undersecretary  of  Defense  for  Research  and 
Engineering  (Acquisition  Policy).  This  office  is  also  responsible  for  the  Defense 
Acquisition  Regulation  (DAR)  (previously  Armed  Service  Procurement  Regula- 
tion, or  ASPR),  the  prunury  source  of  directive  guidance  to  the  contracting  officer  in 
the  creation  and  management  of  the  contract(s).  Finally,  there  is  the  coterie  of 
specialist  communities  that  advise  and  support  the  program  manager  in  the 
interpretation  and  application  of  the  many  thousands  of  MIL-SPECs  and  Mll- 
STDs,  and  the  Air  Force  Plant  Representative  Office  (AFPRO),  Navy  Plant 
Representative  Office  (NAVPRO),  or  other  government  representative  who  pro- 
vides local  surveillance  at  the  contractor's  plant  in  support  of  the  program  office  and 
the  procuring  service. 

Relative  to  tailoring,  the  program  office  can  receive  direction  or  guidance  via  the 
program  manager’s  chain  of  command  and  the  somewhat  independent  channel  of 
DAR/ASPR,  and  assistance  from  the  specialist  communities.  Since  the  preparing 
activities  are  in  almost  all  cases  an  integral  part  of  the  specialist  communities,  and 
since  the  program  manager  usually  is  not  personally  experienced  in  the  many 
specialities  involved,  the  channel  to  him  through  the  specialist  communities  fre- 
quently serves  effectively  as  a third  directive  channel,  even  though  an  official 
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larly  of  the  management  discipline  specifications,  prior  to  issuance  of  the  request  for 
proposal  (RFP).  An  unavoidable  reality,  however,  is  that  the  hulk  of  actual  tailoring 
work  must  he  done  by  the  eontraetor(s),  who  are  the  only  ones  who  have  both  the 
experienced  people  and  the  integral  involvement  with  the  design  efTort  necessary  for 
effective  tailoring.  The  only  other  possible  performers  of  tailoring,  the  military 
specialist  communities,  are  naturally  somewhat  parochially  motivated  not  to  elimi- 
nate or  diminish  the  detailed  provisions  that  are  the  heart  of  their  reason  for 
existence  in  the  first  place.  Also,  they  have  no  direct  involvement  with  the  cost 
pressures  which  are  the  primary  reason  for  tailoring. 

The  tangible  results  of  tailoring,  of  course,  are  changes  to  the  contract  or  other 
formal  documents  defining  the  program  effort.  Thus  the  output  of  the  contractor's 
effort  is  typically  a proposal,  which  it  is  up  to  the  program  office  to  approve  or 
otherwise  dispose  of.  The  specification  and  standard  preparing  activities  and  the 
remainder  of  the  specialist  communities,  although  inherently  ill-adapted  to  the 
actual  accomplishment  of  tailoring,  nonetheless  can  play  an  important  role  in 
structuring  the  specifications  and  standards  in  such  a way  as  to  facilitate  tailoring. 

With  respect  to  the  interpretive  category  of  tailoring,  assuming  that  no  actual 
substantive  change  in  the  intent  of  the  document  is  desired,  the  principal  players 
usually  are  the  contractor  and  his  local  customer  surveillance  office.  The  program 
office  may  be  called  upon  to  ratify  significant  agreements,  however,  or  to  referee 
significant  disagreements. 

Influences  on  the  Central  Participants 

Now  let’s  look  a little  more  closely  at  some  of  the  influences  on  the  program 
office  and  the  contractor  that  play  an  important  role  in  determining  whether 
effective  tailoring  can  or  cannot  take  place. 

Major  influences  on  the  program  manager  pertinent  to  tailoring  include  direc- 
tives coming  down  his  chain  of  command,  his  dependence  on  the  various  specialist 
communities,  particularly  with  regard  to  disciplines  in  which  he  has  little  or  no 
personal  background,  and  the  political  and  other  career  risks  that  are  inherent  in  the 
position  of  program  manager.  Now,  the  kind  of  directives  he  may  get  relative  to 
tailoring,  ranging  from  a simple  "Do  it,"  to  directives  prescribing  in  detail  how  he 
should  do  it  and  how  he  should  document  the  results,  are  likely  to  appear  to  him  as 
constraints,  or  hurdles  to  be  cleared,  i.e.,  one  more  set  of  problems  to  contend  w ith, 
on  top  of  the  horrendous  stack  he  already  has  Unfortunately,  the  specialist 
communities  on  which  the  program  manager  must  depend  may  seem  toconlribule  to 
the  problem  rather  than  to  the  solution,  because  of  their  ow  n natural  motivations  not 
to  tailor  out  any  of  the  provisions  that  seem  so  important  to  them  Finally,  as  a built- 
in  career  risk  that  goes  with  the  job.  there  is  the  ever-prc  sent  possibility  of  high-level 
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criticism  for  some  alleged  failure  of  the  program  manager  to  adequately  promote  or 
protect  the  interests  of  the  government  in  his  dealings  with  the  contractor.  The 
almost  certain  net  result  is  some  degree  of  reluctance  to  depart  from  the  safety  and 
convenience  of  full-whammy  application  of  all  the  officially  approved  specifications 

I and  standards. 

Turning  now  to  the  contracting  officer,  pertinent  major  influences  on  him  are  his 
semi-autonomy  from  the  program  manager,  the  legal  authority  of  DAR/ASPR,  and 
a deeply  ingrained  concept  of  "consideration."  The  independence  provided  the 
contracting  officer  by  his  warrant  serves  to  insulate  his  contracting  and  financial 
responsibilities  from  the  totality  of  overall  program  goals.  On  the  other  hand,  he  is 
unequivocally  committed  to  the  legally  binding  DAR/ASPR.  which  has  been 
inherently  conservative  because  of  the  nature  of  the  committee  w hich  until  recently 
has  been  responsible  for  its  care  and  feeding.  And  at  an  even  more  fundamenta I level, 
the  contracting  officer  has  been  trained  from  the  beginning  to  regard  contracting  as 
an  adversary  proceeding  in  which  concessions  should  be  made  only  if  there  is  an 
equally  valuable  quid  pro  quo  Thus,  elimination  of  any  contractual  requirement  by 
tailoring,  regardless  of  the  actual  utility  of  that  requirement,  would  automatically 
appear  to  him  (and  his  functional  superiors)  to  call  for  "consideration"  from  the 
contractor.  The  result  of  these  various  influences  is  a deep-seated  bias  against 
permitting  the  contractor  to  benefit  by  tailoring  actions,  particularly  any  that  might 
be  interpreted  as  "letting  the  contractor  off  the  hook." 

Reluctance  about  tailoring  on  the  part  of  the  program  manager,  plus  orientation 
of  the  contracting  officer  against  permitting  the  contractor  to  benefit  thereby,  will 
tend  to  produce  strong  anti-tailoring  motivation  on  the  part  of  the  contractor  as  well 
Although  industry  inherently  favors  tailoring  because  of  the  greater  flexibility  and 
efficiency  it  promises,  the  most  fundamental  objective  is  to  stay  in  business  and  make 
a profit.  That  requires,  first  of  all,  winning  competitions,  and  that  generally  means 
offering  most  nearly  what  the  program  office  really  wants.  If.  as  usual,  the  RFP  calls 
out  a collection  of  MIL-SPECs  and  Mil. -STDs,  most  proposals  w ill  advertise  full 
compliance  therewith  rather  than  risk  being  rejected  as  non  responsive  Even  if  the 
RFP  explicitly  encourages  tailoring,  the  proposals  will  still  feature  full  compliance  to 
the  significant  specifications  if  the  competitors  sense  that  this  is  what  the  program 
office  really  wants.  After  all.  protests  seldom  win  competitions.  And  even  after  the 
contract  is  awarded,  there  will  be  little  contractor  enthusiasm  for  tailoring  if  he  is  not 
permitted  to  share  in  the  savings  that  result  Another  channel  of  customer  influence 
is  the  local  government  surveillance  agency  (AFPRO,  NAVPRO.  etc.).  This 
influence  tends  also  to  he  anti-tailoring  to  the  extent  that  it  seeks,  via  the  Air  Force’s 
Contractor  Management  System  Evaluation  Program  or  similar  means,  to  judge 
contractor  management  adequacy  and  compliance  independently  of  contractual 
requirements. 
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The  Key  Recommendation:  Accurate  Perception 

Analysis  of  the  preceding  discussion  of  anti-tailoring  influences  reveals  that  they 
stem  overwhelmingly  from  the  perception  that  tailoring  means  giving  up  something; 
that  it  means  retreating  from  what  we  really  would  like  to  have  because  we  decide 
that  that  last  increment  of  goodness  isn’t  worth  what  it  costs.  Reflecting  the  views  of 
a broad  segment  of  industry,  however,  I suggest  that  this  perception  is  wrong , that 
what  we  usually  are  seeking  via  tailoring,  just  as  in  the  case  of  value  engineering,  is  a 
product  better  suited  to  its  application,  not  an  inferior  product. 

In  Figure  3 we  see  this  point  pursued  further.  The  left-hand  column  contains 
phrases  that  I believe  characterize  the  most  commonly  held  perception  of  tailoring, 
particularly  as  applied  to  general  design  and  product  discipline  specifications.  The 
opposite  column,  on  the  other  hand,  contains  the  corresponding  phrases  that  I think 
accurately  characterize  what  most  meaningful  tailoring  actually  is  all  about.  The 
first  phrase  in  that  column  lies  at  the  very  heart  of  the  issue.  For  the  majority  of  the 
potential  cost  driver  specifications  and  standards  identified  in  the  Shea  Panel 
report,*  indiscriminate  full  application  to  a real  program  would  not  be  ideal  but 
would  in  fact  be  excessive  and,  in  some  cases,  positively  undesirable.  The  goal  of 
tailoring  is  therefore  properly  perceived  not  as  invocation  of  the  minimum  tolerable 
selection  from  a set  of  requirements,  all  of  which  are  desirable,  but  rather  invocation 
of  the  pertinent  or  optimum  requirements  from  a set,  some  of  w hich  are  and  some  of 
which  are  not  meaningfully  applicable  to  the  particular  program.  Correspondingly, 
the  contractor  actions  desired  should  be  viewed  not  in  the  negative  light  of  seeking 
deviations  and  waivers  to  already  invoked  (and  therefore  presumably  desirable) 
specifications,  but  rather  in  the  positive  light  of  recommending  the  optimization  of 
the  specification  structure. 

Perhaps  a more  incisive  way  of  putting  the  above  thoughts  is  to  emphasize  that 
we  are  not  urging  the  program  offices  to  acquire  marginal  systems,  nor  to  function 
with  marginal  management  controls.  Rather,  we  are  asking  that  they  acquire 
systems  that  are  optimum  for  their  applications,  avoiding  in  the  process  require- 
ments that  are  inappropriate,  excessive,  or  perhaps  actually  counterproductive. 

This  point  is  of  crucial  importance,  because  the  numerous,  strong  anti-tailoring 
influences  on  the  program  manager  and  contracting  officer  are  very  largely  elimina- 
ted by  acceptance  of  the  positive  perception  summarized  in  the  right-hand  column  of 
Figure  3.  It  is  vigorously  urged  that  a major  campaign  be  mounted  to  get  this 
contrasting  perception  recognized  and  promulgated  throughout  the  various  commu- 
nities (including  the  Congress)  that  are  involved  in  defense  materiel  acquisition. 


•Defense  Science  Board  Reporl.  Report  of  the  Task.  Force  on  Specifications  and  Standards,  Dr 
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Suggestions  Concerning  Contractor  Motivation 

From  the  earlier  analysis  of  influences,  it  is  possible  to  develop  recommendations 
for  specific  actions  that  the  program  office  can  take  to  stimulate  meaningful  tailoring 
inputs  on  the  part  of  its  contractors),  assuming  it  understands  and  accepts  the  real 
significance  of  tailoring  as  just  presented.  These  suggestions  are  aimed  at  eliminating 
the  contractor’s  fear  of  penalty,  both  directly  and  by  clearly  demonstrating  the 
program  office’s  positive  orientation.  They  are  also  intended  to  provide  positive 
motivation  for  the  desired  action. 

In  RFPs,  the  fear  of  being  held  nonresponsive  can  be  eliminated  by  not  providing 
a pre-selected  list  of  specifications  to  which  tailoring  recommendations  could  appear 
nonresponsive.  An  even  more  positive  signal  can  be  given  by  indicating  that  the 
quality  of  tailoring  recommendations  or  evidence  of  tailoring  capability  will  be 
factors  in  proposal  evaluation. 

In  contracts,  the  fear  of  penalty  for  tailoring-type  changes  can  be  minimized  by 
referring  to  the  desire  for  specification  optimization  (positive  concept)  rather  than  to 
the  possible  consideration  of  deviations  and  waivers  (negative  concept).  That  fear 
can  be  completely  eliminated  by  providing  specifically  for  contractor  sharing  in  the 
savings  (e.g.,  via  the  established  incentive  pattern).  The  most  positive  approach  to 
tailoring,  of  course,  is  to  include  it  as  a specific  task  in  the  statement  of  work.  On 
major  system  acquisitions,  serious  consideration  should  be  given  to  a funded 
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contract  definition  phase  in  which  initial  tailoring  of  key  general  design  and  product 
discipline  specifications  receives  significant  attention. 

The  recent  DOD  Directive  5000.35,  Defense  Acquisition  Regulatory  System, 
offers  a channel  for  communicating  DOD  policies  and  instructions  concerning 
tailoring  directly,  promptly,  and  undilutedly  to  the  program  offices.  It  is  recom- 
mended that  appropriate  sections  of  the  DAR  be  very  carefully  written  to  reflect 
unequivocally  the  basic  perception  of  tailoring  outlined  here  and  to  provide  for  the 
suggested  RFP  and  contract  approaches. 

Organizational  Recommendation 

Finally,  we  have  an  organizational  recommendation  for  facilitating  a sound  focus 
on  and  effective  implementation  of  tailoring  within  the  various  services.  This 
recommendation  stems  from  the  observation  that  the  basic  objective  of  tailoring  is  to 
save  money  while  leaving  undiminished,  or  even  enhancing,  the  utility  of  the 
product.  The  fundamental  approach  is  to  assess  the  pertinence  of  the  various  existing 
specifications  and  standards,  and  of  individual  segments  thereof,  to  the  particular 
application  under  consideration  and  to  make  cost/benefit  trade-offs  as  appropriate. 

Now,  the  specialist  communities  that  generate  and  provide  consultation  concern- 
ing the  myriad  of  MIL-SPECs  and  MIL-STDs  generally  are  not  well  adapted  to 
making  this  kind  of  trade-off.  On  the  other  hand,  this  approach  is  precisely  that 
which  underlies  the  well-established  disciplines  of  value  engineering  and  design-to- 
cost.  As  a matter  of  fact,  tailoring  can  be  accurately  characterized  as  the  application 
of  the  value  engineering  and  design-to-cost  disciplines  to  specifications  and  stan- 
dards. This  leads  to  a recommendation  that  the  responsibility  for  promoting, 
facilitating  and  overseeing  tailoring  within  each  of  the  services  be  assigned  to  a 
Directorate  of  Acquisition  Cost-Effectiveness,  which  would  also  be  responsible  for 
value  engineering,  design  to  cost,  and  related  cost-effectiveness  factors. 

Summary 

The  first  recommendation  presented  here,  that  of  working  to  develop  an 
accurately  positive  perception  of  tailoring  throughout  the  defense  acquisition  com- 
munity, is  by  far  the  most  important.  In  fact,  it  is  the  basis  for  the  successful 
realization  of  the  specific  actions  recommended.  If  this  positive  perception  can  be 
developed  (it  can),  and  if  the  other  recommendations  are  implemented,  the  technique 
of  tailoring  will  cease  to  be  discussed  as  a technique  and  will  take  its  place  alongside 
value  engineering  and  design-to-cost  as  routine  concepts  in  the  quest  for  acquisition 
effectiveness.  | 


SURVEILLANCE  OF 
DEFENSE  PROGRAMS: 
THE  INDUSTRY  ROLE 

Irving  J.  Sandler 

The  question,  “How  much  government  surveillance  and  how  much  regulation  of 
defense  industry  is  enough?"  is  one  that  must  be  continually  addressed  by  the 
Defense  Contract  Audit  Agency  (DCAA).  Over  the  years  one  of  the  greatest 
challenges  facing  the  agency  has  been  to  determine  how  best  to  utilize  available 
resources  in  fulfilling  DCAA  audit  responsibilities  under  existing  public  laws  and 
procurement  regulations.  This  has  required  DCAA  to  continually  assess  its  priori- 
ties and  direct  its  efforts  to  those  areas  believed  to  carry  the  highest  risk. 

Determining  the  amount  of  resources  which  should  be  devoted  to  an  audit  is  a 
problem  facing  both  government  and  non-government  auditors.  The  problem  is 
made  even  more  complex  because  of  the  pressures  for  increased  emphasis  on 
regulatory  controls  that  have  come  from  many  quarters,  including  the  press,  the 
legislature,  the  Securities  and  Exchange  Commission,  and  introspective  examina- 
tions within  the  accounting  community  itself.  One  might  conclude  that  this  is  a 
problem  with  little  or  no  solution  since  on  the  one  hand  there  is  a demand  for  more 
surveillance,  while  on  the  other  there  are  insufficient  resources  to  carry  it  out.  But 
there  is  a solution,  and  to  a large  degree  it  rests  with  and  can  be  influenced  by  defense 
contractors  themselves. 

Audit  Standards 

It  may  be  helpful  to  highlight  the  auditing  standards  that  form  the  framework 
within  which  DCAA  auditors  are  expected  to  operate.  Essentially,  DCAA  auditors 
follow  the  same  basic  standards  of  auditing  as  specified  by  the  American  Institute  of 
Certified  Public  Accountants  (AICPA).  The  Defense  Contract  Audit  Manual 
requires  that  the  audit  be  performed  by  persons  having  adequate  technical  training 
and  proficiency  as  auditors.  Ti.ey  must  be  independent  of  influence  or  control  by 
others  and  are  required  to  exercise  due  professional  care  in  performing  the  audits  and 
preparing  the  reports.  The  standards  established  by  the  AICPA  and  DCAA  for  field 
work  emphasize  the  need  both  for  planning  and  supervision  of  the  audits,  and  a 
proper  evaluation  of  internal  control  to  determine  the  degree  of  reliance  that  can  be 
placed  on  a company’s  accounting  system.  Beyond  these  basic  standards,  however, 
are  the  more  elaborate  standards  promulgated  by  the  General  Accounting  Office 
(GAO)  to  which  DCAA  must  also  adhere.  These  standards  require  the  agency  to  (1) 
look  into  the  financial  and  compliance  aspects  of  contractor  claims  to  determine 
whether  financial  operations  are  properly  conducted,  whether  financial  reports  of  an 
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audited  entity  are  presented  fairly,  and  whether  the  entity  has  complied  with 
applicable  law>  and  regulations;  and  to  (2)  look  at  how  the  entity  is  managing  or 
utilizing  its  resources  (personnel,  property,  space,  etc.)  This  requires  that  the  agency 
determine  causes  of  any  inefficiencies  or  uneconomical  practices,  including  inade- 
quacies in  management  information  systems,  administrative  procedures,  or  orga- 
nizational structure. 

With  this  very  brief  overview  of  auditing  standards,  let  us  now  concentrate  on  the 
audit  standard  that  deals  with  the  degree  to  which  an  auditor  may  rely  on  a 
contractor’s  system  of  internal  control.  DCAA  prefers  that  management  take  the 
initiative  to  monitor  its  own  operations  to  assure  efficient  and  economical  perfor- 
mance. Under  such  circumstances,  DCAA  involvement  can  be  significantly  re- 
duced, and  from  industry’s  perspective  government  surveillance  can  be  reduced.  If 
the  agency  can  place  a high  degree  of  reliance  on  a contractor's  system  of  internal 
control — that  is,  if  its  estimated  costs  are  reasonable,  if  the  organization  is  efficiently 
managed,  and  if  the  cost  accounting  system  provides  for  the  screening  of  unallowable 
expenses  and  produces  equitable  allocations — then  the  risks  to  the  government  are 
considerably  lessened  and  the  scope  of  DCAA  audit  reviews  can  be  adjusted.  Thus, 
the  answer  to  the  question,  “How  much  surveillance  is  enough?"  depends  a great 
deal  on  the  degree  to  which  industry  maintains  surveillance  of  itself.  This  is  the  goal 
that  government  and  industry  should  be  jointly  striving  to  reach. 

One  of  the  best  internal  controls  is  an  internal  audit  function.  Once  again,  DCAA 
position  on  the  use  of  the  work  of  industry’s  internal  auditors  is  consistent  with  that 
of  the  AICPA.  While  the  work  of  the  contractor’s  internal  auditors  is  not  considered 
a complete  substitute  for  the  DCAA  audit,  it  is  taken  into  consideration  by  the 
Agency  when  determining  the  nature,  timing,  and  extent  of  its  own  auditing 
procedures. 

The  mere  presence  of  a contractor  internal  audit  group  does  not  in  itself  mean 
that  DCAA  would  rely  on  the  work  of  that  group  in  determining  the  scope  of  its 
reviews.  The  agency  would  first  review  the  contractor’s  internal  audit  function  to 
determine  its  overall  mission  and  responsibilities.  The  agency  is  particularly  inter- 
ested in  several  areas,  such  as  work  performed  by  the  internal  auditors  that  could 
affect  planned  DCAA  audits,  the  competence  of  the  contractor's  staff,  and  the 
independence  of  the  internal  audit  organization.  Significant  in  this  regard  are 
qualifications  of  the  contractor’s  internal  audit  staff,  their  training,  and  their 
supervision.  Additionally,  the  objectivity  of  the  internal  auditors  is  an  important 
consideration.  Their  reporting  level  in  the  organization  should  be  high  enough  to 
permit  them  to  work  independently  of  those  responsible  for  the  functions  being 
audited. 

As  a final  point,  arrangements  must  be  made  with  the  contractor's  internal  audit 
staff  for  access  to  their  audit  programs,  working  papers,  and  reports.  Hence,  if  the 
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contractor’s  internal  auditors  are  competent  and  they  permit  the  government  to 
review  their  work  to  make  sure  that  their  conclusions  can  be  relied  upon,  govern- 
ment surveillance  can  be  reduced. 

Public  Law  87-653 — Truth  in  Negotiations 

Implementation  of  Public  Law  87-653  by  the  Department  of  Defense  created  a 
special  area  of  DCA  A responsibility  for  surveillance  of  defense  contractors.  This  law 
was  enacted  in  1962  as  a result  of  GAO  reports  of  overpricings  disclosed  in  the 
negotiation  process.  It  is  important  to  remember  that  this  law  was  enacted  despite 
the  fact  that  its  provisions  were  already  substantially  covered  by  the  Armed  Services 
Procurement  Regulation,  now  called  the  Defense  Acquisition  Regulation  (DAR).  It 
was  because  of  the  reported  noncompliance  with  such  existing  regulations  that 
Congress  deemed  it  necessary  to  put  the  full  effect  of  the  law  behind  pricing 
requirements.  Public  Law  8 7-653  gives  the  government  the  right  to  adjust  the 
contract  price  when  that  price  is  based  on  inaccurate,  incomplete,  or  noncurrent  cost 
or  pricing  data.  DOD  assigned  contractor  surveillance  of  this  program  to  DCAA, 
who  carries  it  out  by  sampling  defense  contract  awards.  Since  DCAA  began  this 
effort,  there  has  been  an  overall  reduction  in  the  incidence  of  defective  pricing,  and 
the  Agency’s  selection  process  has  been  modified  to  emphasize  high-risk  pricing 
actions. 

The  overall  implementation  of  this  program  has  worked  satisfactorily,  but  it  is  a 
remedial  approach  at  best  and  is  fraught  with  the  usual  burdensome  process  of 
administrative  settlement  and  appeal  whenever  a case  of  overpricing  is  suspected.  An 
obviously  more  productive  use  of  both  contractor  and  government  resources  would 
be  to  shift  emphasis  from  a remedial  to  a preventive  mode;  that  is,  to  have  industry 
build  in  controls  to  assure  the  accuracy,  currentness,  and  completeness  of  bid 
proposal  submissions  at  the  time  the  bid  proposals  are  negotiated.  The  procedure  is 
often  relatively  simple  involving  a survey  to  detect  the  source  of  any  deficiencies.  An 
example  of  a common  problem  in  this  area  involves  timing.  Current  information  on 
the  latest  material  and  parts  prices,  and  budgetary  information  on  labor  or  indirect 
costing  rates  may  take  an  inordinate  amount  of  time  to  flow  through  the  "system"  to 
the  contract  negotiator.  What  is  considered  a reasonable  time  period  to  update 
information  varies  with  the  materiality  and  sensitivity  of  the  item  involved.  A major 
item  affecting  cost  may  have  to  be  updated  on  the  same  day.  Control  procedures  can 
be  revised  to  correct  deficiencies,  generally  with  little  interruption  or  additional  cost. 
Where  these  controls  exist  in  industry,  DCAA  and  even  GAO  can  reduce  surveil- 
lance to  the  point  of  only  making  test  checks  to  assure  continuing  reliability. 

Unallowable  and  Unclaimed  Cost  Identification  System 

An  area  that  receives  widespread  publicity  and  which  continues  to  plague  DOD 
and  defense  industry  involves  such  expenses  as  gratuities,  entertainment,  etc.  These 
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costs  are  expressly  unallowable  and  many  are  not  claimed  by  contractors;  however,  a 
problem  arises  because  of  the  considerable  effort  required  to  determine  directly 
associated  costs.  The  answer  lies  not  with  more  government  surveillance,  but  rather 
with  some  type  of  systematic  procedure  to  be  used  by  contractors  in  accounting  for 
these  costs.  The  DAR  Council  recognizes  the  sensitivity  of  this  problem  and  is 
working  on  guidelines  to  resolve  it. 

Fraud 

!t  is  difficult  for  an  auditor  not  to  talk  about  fraud.  The  Contract  Audit  Manual 
outlines  DCAA’s  responsibility  for  detecting  fraud.  DCAA  auditors  are  expected  to 
be  alert  for  situations  or  transactions  that  may  involve  fraud.  Common  examples  of 
such  activities  include  falsification  of  documents  such  as  time  cards  or  purchase 
orders;  charging  of  personal  expenses  to  government  contracts;  submission  of  claims 
for  services  not  performed  or  materials  not  delivered;  intentional  mischarging  or 
misallocation  of  costs;  deceit  by  suppression  of  the  truth;  regulatory  or  statutory 
violations  such  as  bribery,  theft,  gratuities,  graft,  or  conflict  of  interest.  The  auditor's 
alertness  and  tests  of  procedures  and  transactions,  combined  with  the  operation  of 
the  contractor's  own  internal  controls,  should  reasonably  insure  that  material  fraud 
or  other  unlawful  activity  is  disclosed.  However,  there  is  growing  pressure  from  the 
Department  of  Justice  for  government  auditors  to  give  increased  attention  to 
potential  wrongdoing  within  the  contract  environment.  The  Department  has  ex- 
pressed concern  that  often  too  little  thought  is  given  to  the  soundness  of  a company's 
system  of  internal  control  to  minimize  the  potential  for  fraud. 

It  should  be  recognized  that  the  amount  of  audit  required  to  insure  that  all  fraud 
is  discovered  is  prohibitive,  since  it  would  entail  examining  every  transaction. 
Therefore,  simply  from  a pragmatic  point  of  view,  the  Defense  Contract  Audit 
Agency  must  review  and  rely  upon,  where  possible,  the  effectiveness  of  the  system  of 
internal  control  maintained  by  the  contractor.  Weaknesses  in  a contractor’s  internal 
control  can  be  an  indication  of  poor  management  and  recordkeeping,  or  of  an 
intentional  coverup  for  fraud.  When  such  weaknesses  are  found,  surveillance  efforts 
are  expanded.  Conversely,  when  a contractor’s  systems  are  considered  to  be 
adequate  and  tests  show  no  deficiencies,  the  level  of  surveillance  can  be  reduced. 
Agency  experience  indicates  that  defense  contractors  are  willing  to  listen  and  many 
times  adopt  DCAA  recommendations  to  improve  upon  their  systems  of  internal 
control  to  further  minimize  the  potential  for  fraud.  Because  of  this  environment, 
DCAA  has  been  able  to  spend  less  time  pursuing  suspected  fraud. 

Opportunities  for  Cost  Avoidance 

It  has  been  emphasized  that  industry  can  help  itself  reduce  government  surveil- 
lance by  helping  DCAA.  This  may  seem  a bit  incongruous,  because  in  order  to 
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decrease  government  surveillance.  industry  will  have  lo  incur  some  additional  costs 
However,  evidence  has  shown  that  good  internal  controls  pay  lor  themselves,  often 
many  times  over  I'he  most  common  example  would  he  when  adequate  controls 
prevent  an  instance  of  fraud  Hut  the  potential  for  even  greater  payback  lies  with  cost 
avoidance  opportunities  made  possible  by  the  elimination  of  inefficient  and  uneco- 
nomical practices 

The  Defense  Contract  Audit  Agency  has  achieved  excellent  results  m recent 
years  m reviewing  contractor  performance  for  efficiency  and  economy  of  operations 
I'lie  techniques  employed  do  not  differ  from  those  which  may  In-  used  by  industry 
internal  audit  organizations  in  assessing  their  own  operations 

A new  regulation,  DAK  '0  l(XX).  has  been  developed  to  monitor  contractoi 
costs  along  these  lines  I join  the  vantage  point  of  contract  auditors,  this  program  is 
not  new  but  is  ralhei  a compilation  of  some  of  the  better  aspects  of  previously 
existing  procedures  for  seeking  cost  avoidance  opportunities 

I he  effective  performance  of  these  reviews  requires  the  cooperation  of  govern- 
ment audit  and  technical  personnel  Where  contractors  are  willing,  these  reviews 
could  also  be  accomplished  with  the  cooperation  of  contractoi  technical  personnel 
Current  emphasis  is  on  such  areas  as  production  scheduling  and  control,  energy 
conservation,  facilities  management,  and  quality  assurance 

One  of  DC  A A s most  recent  and  more  significant  findings  lor  potential  savings  is 
in  the  area  of  computet  graphics  I'he  most  feitile  areas  for  use  of  this  technology  are 
m the  design  and  drafting  field  A single  audit  of  a major  contractor  revealed  that  as 
much  as  Sts  million  to  $7  million  could  be  saved  annually  if  computers  rather  than 
manual  methods  were  used  in  preparing  production,  tooling,  and  facilities  draw  ings 
Audit  of  another  contractor  indicated  that  expanded  use  of  computer  graphics  could 
save  between  SMX)  thousand  and  $1  X million  annually  It  has  been  suggested  to 
DOD  that  ways  he  considered  to  encourage  more  contractors  to  use  computet 
graphics 

Because  reviews  of  economy  and  efficiency  continue  to  reveal  areas  for  substan- 
tial payoff.  IX'AA  believes  it  necessary  to  continue  these  audits  even  though 
industry  sometimes  views  them  as  increased  surveillance 

Summary 

For  the  most  part,  the  degree  to  which  the  Defense  Contract  Audit  Agency 
audits  industry  depends  on  industry  itself  A lullei  understanding  of  the  contract 
audit  mission,  the  audit  standards  adhered  to.  and  the  intricacies  of  certain  specific 
areas  of  audit  concern  such  as  fraud  and  defective  pricing,  can  lead  to  improved 
internal  controls  bv  defense  industry  I'he  mutual  interests  of  government  and 
industry  can  often  be  met  though  satisfactory  internal  company  controls  rathei  than 
more  government  surveillance  l o this  end,  it  seems  cleat  that  industry  can  take  an 
active  i ole  in  answering  the  question.  "How  much  is  enough'*"  || 
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This  discussion  presumes,  for  the  sake  of  brevity,  that  pre-award  procurement 
activities  were  properly  accomplished.  Our  concern  here  is  with  the  indications  and 
alternatives  in  the  post-award  phase  of  procurement.  "Now  that  we're  married,  how 
can  we  live  together?"  might  be  the  stated  question. 

The  Con t mental  Relationship 

As  noted  previously,  a mutual  concern  for  the  success  of  a contract  may  lead  to  a 
feeling  of  common  purpose  or  "partnership"  between  the  contract  management 
teams  m industry  and  government.  While  the  government  works  closely  with  a 
contractor  during  the  contract  administration  phase,  this  relationship  by  necessity 
remains  at  "arm's  length  " Two  reasons  dictate  this  relationship.  First,  the 
government  contractor  relationship  is  normally  on  a continuum,  for  old  contracts 
are  being  completed  while  new  ones  are  beginning.  Consequently,  a delicate 
relationship  exists  to  avoid  any  appearance  of  misconduct  on  the  part  of  the 
government.  Thus  a formal  business  relationship  is  required,  even  when  a close 
harmonious  working  association  may  exist  Second,  when  contractor  and  govern- 
ment enter  into  a contract,  an  obligation  to  contractual  fulfillment  is  incurred.  Any 
failure  to  meet  the  terms  of  the  obligation  may  result  m a penally  under  the  law,  l'he 
effect  then  becomes  a careful  assurance  of  compliance  by  each  party.  Distrust  is  not 
implied  but  mutuality  of  concern  is 

The  relationship  between  the  government  customer  and  the  private  seller  is 
documented  by  the  contract  However,  this  relationship  dives  not  extend  beyond  the 
prime  contractor  to  his  suppliers,  variously  known  as  vendors  or  subcontractors.  Yet 
the  impact  of  suK  ntract  or  vendor  firms  can  make  or  break  the  program. 

While  the  government  lacks  a direct  contractual  relationship  with  the  vendors, 
often  there  is  a requirement  for  government  quality  assurance  inspectors  to  witness, 
verify,  or  inspect  the  work  done  Hits  requirement  is  commonly  made  known  by  a 
letter  delegating  specific  responsibilities  from  the  prime  contract  administration 
activity.  For  example,  an  Air  Force  plant  representative  office  (AFI’ROi  may 
delegate  inspection  tasks  to  a Defense  Contract  Administration  Service  Management 
Area  (DC  ASM  A I or  vice  versa 

It  is  through  this  channel  that  government-vendor  involvement  occurs.  How- 
ever, the  management  of  the  subcontractor  is  explicitly  the  responsibility  of  the 
prime  contractor,  not  the  government,  although  the  government  is  available  to  assist 
the  prime  contractor  upon  request  In  special  situations,  the  vendor  program 
manager  may  receive  assistance  in  a non-directive,  consultative  manner  In  any  case, 
principal  contract  management  emphasis  is  appropriately  with  the  prune  contractor, 
who  is  paid  implicitly  to  deliver  management  expertise  in  addition  to  hardware, 
software,  data,  and  services. 
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Obviously,  the  buying  office  will  also  need  to  document  its  desires  regarding 
contract  management  with  the  plant  representative.  This  essential  team  relationship 
can  help  to  assure  contract  success  by  emphasising  responsibilities  in  the  letter  of 
delegation,  and  may  possibly  result  in  a memorandum  of  agreement,  which  assigns 
contract  administration  and  clarifies  complex  issues  and  responsibilities  of  an 
ongoing  nature.' 

What  Can  Go  Wrong? 

Having  designated  the  needs  and  responsibilities  in  the  contract  administration 
delegations,  one  might  expect  success  to  be  inevitable.  Yet  in  monitoring  contract 
deliveries  and  progress  toward  other  post-award  milestones,  we  find  a recurring  set 
of  problems.  An  effective  monitoring  system  may  reveal  that: 

1.  The  contractor  or  vendor  did  not  understand  the  contractual  requirements 
when  his  proposal  or  bid  was  submitted; 

2.  The  data  package  was  inadequate  to  convey  the  requirement,  necessitating 
clarification  and  causing  delay; 

.V  The  complexity  surpasses  the  capabilities  of  the  contractor’s  skills,  requiring 
reconsideration  of  the  earlier  (prior  to  contract  award)  make-or-buy  decision; 

4.  Materials  are  not  available  when  required,  or  at  all; 

5.  The  contract  conflicts  with  prior  commitments  in  contractor's  facility; 

b.  The  contractor’s  plant  lacks  physical  capacity  to  achieve  required  deliveries; 

7.  The  financial  posture  of  the  contractor  impedes  timely  delivery,  perhaps 
because  his  suppliers  demand  cash  payment; 

8.  Production  planning  is  inadequate; 

4.  Quality  assurance  is  inadequate  to  produce  materials  in  conformance  with 
specifications; 

10.  Excusable  delays  preclude  timely  delivery  (e  g.,  strikes,  floods,  tornadoes, 
etc.); 

11.  Government  furnished  material  or  equipment  is  delivered  late,  or  incom- 
plete. or  is  of  improper  configuration 


The  problems  cited  above  are  neither  mutually  exclusive  nor  collectively  exhaus- 
tive. Yet.  they  suggest  the  difficulties  which  may  arise  after  the  signing  of  the 
contract  and  the  often  too-brief  "honeymoon"  which  follows.  While  most  of  the 
difficulties  outlined  may  be  avoided  by  effectively  accomplishing  a pre-award  survey 
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or  in  pre-award  conferences.  tiv>  often  the  problems  go  undetected  until  it  is  tune  for 
the  contractor  to  perform  by  delivering  the  required  items  or  services. 

How  Can  1 Fix  If.' 

What  alternatives  exist  to  resolve  such  situations  threatening  performance  of  the 
contract?  At  the  risk  of  emphasizing  the  obvious,  it  must  he  said  that  candid 
communications  in  several  packages  tailored  to  the  circumstance  can  help.  Choices 
exist.  Each  option  is  designed  to  produce  timely  delivery  of  a product  w ith  suitable 
quality  for  the  price  agreed  upon.  Further,  each  option  helps  to  protect  the  rights  of 
the  government  under  the  terms  of  the  contract.  Thus.  each  action  taken  should  be 
carefully  documented  in  the  official  contract  file  (as  well  as  in  the  office  taking  the 
action  it"  a separate  working  file  is  maintained),  for  today's  actions  have  a habit  of 
becoming  involved  in  tomorrow's  problems. 

Usually,  awareness  comes  in  steps  Trouble  can  be  anticipated  if  the  symptoms 
are  recognized  and  properly  addressed  Consequently,  the  following  courses  of 
action  represent  selective  communicative  responses  or  initiatives  which  may  be 
employed.  They  range  in  intensity  from  a phone  call  to  termination  for  default 
action  w hich,  w hile  being  close  to  the  ultimate  weapon,  unfortunately  fails  to  till 
the  need  for  required  giXKls  or  set  s ices 

Three  categories  of  communication  may  be  used  telephone  communication, 
written  communication,  and  meetings  Each  of  these  has  inherent  advantages  and 
disadvantages  Management  judgment  in  light  of  facts,  inferences,  and  perceptions  is 
essential  in  their  application,  but  each  of  the  following  can  contribute  to  correction 
of  contract  performance  problems 

Using  the  Phone 

Quick,  flexible,  inexpensive,  well-planned  telephone  contacts  may  correct  a 
troublesome  situation  One's  personality  may  be  effectively  projected,  and  results 
may  be  sensed  immediately  in  some  cases  Also,  one's  call  may  be  transferred  from 
desk  to  desk  until  the  proper  people  arc  personally  contacted  tor  resolution  of  the 
issue  While  the  relative  cost  of  using  the  telephone  is  minimal,  (he  results  can  often 
be  immediate.  When  the  response  is  inadequate,  the  problem  can  be  elevated  quickly 
and  appropriately 

For  example,  by  using  the  telephone,  in-depth  understanding  and  discussion  of 
troublesome  issues  can  be  effected  Dialogue  with  the  administrative  contracting 
officer  ( ACO)  and  his  clerks  can  achieve  far  greater  insight  into  contractual  progress 
than  reliance  solely  upon  a routine  "Production  Progress  Report"  (DD  Form  .175) 
The  ACO's  depth  of  knowledge,  interest,  and  involvement  can  be  measured  and 
redirected  when  it  is  determined  to  be  inadequate  Discussions  with  the  production 
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Remember.  however,  that  documentation  memoraiula  are  essential  to  preserve 
an  accurate  "memory”  of  agreements,  open  items,  opinions,  and  insights  developed 
while  using  the  telephone  Follow-up  correspondence  or  documentation  is  ollen 
required  to  record  oral  transactions,  for  the  mind  is  a fickle  servant  with  restricted 
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Written  documentation  of  oral  requests,  commitments,  and  discussions  can  pay 
handsome  dividends  as  issues  evolve  While  such  efforts  are  somewhat  more  costly 
than  telephone  contacts,  they  can  invoke  internal  coordination,  higher  authority 
attention,  and  careful  documentation  of  the  program  office  position  Thus,  while 
hutlding  the  “corporate  memory."  a consensus  of  broad  support  is  also  developed 

A formal  definition  of  contract  administration  expectations  and  responsibilities 
is  usually  achieves!  by  issuing  a letter  of  delegation  and  executing  a memorandum  of 
agreement  in  complex  situations/  delegating  duties  between  the  buying  activity  and 
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specialist  who  wrote  the  production  progress  report  can  be  enlightening.  It  is  often 
the  production  specialist  who  has  the  best  understanding  of  the  contractor's 
circumstances,  he  seises  as  the  ACO's  eves  and  ears. 

Also,  the  level  of  contract  administration  service  (C  AS)  interest  can  be  senses! 
through  careful  attention  to  what  is  said,  and  how.  Should  a discussion  by  telephone 
suggest  a low  priority  of  attention  to  a problem,  the  buying  office  can  request 
increases!  surveillance,  weekly  status  reports.  or  more  stringent  measures  by  the 
CAS.  l'he  program  office  can  become  a "squeaking  w heel"  by  effectively  using  the 
phone  to  develop  personal  relationships  and  to  solicit  commitments  to  rrsivlvc  and 
prevent  problems.  Supporting  the  CAS  rs'le  as  on-the-scene  representative,  or  as  an 
extension  of  the  buying  office,  can  pay-  dividends  to  the  buying  office 

When  the  contractor  employs  hval  representatives,  contact  with  them  may  be 
appropriate  fhese  contacts  can  provide  noeslexl  assistance  and  insight  far  quicker  in 
some  instances  than  could  be  obtained  through  written  communication  I’his  avenue 
may  be  as  effective  as.  or  superior  to.  calling  the  contractor  directly  at  lus  main 
operating  location,  since  the  local  representative  often  has  more  flexibility  and 
dexterity  in  working  within  the  contractor's  organization.  Caution  must  be  exercises! 
to  keep  the  contract  administration  activity  informed  when  such  contacts  are  made 
directly  to  the  prime  contractor. 

Further,  trade-offs  and  work-around  alternatives  can  be  explores!  effectively  by 
telephone  on  a personal  or  conference  basis  Consequently  , the  telephone  is  quite  a 
cost-effective  (<vol  Its  use  in  an  age  of  rest  rides!  travel  budgets  is  also  a welcome 
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the  contract  administration  activity,  as  suggested  earlier  Having  done  this,  the 
management  assistance  letter  is  a useful  escalation  of  communications  between 
operatives  The  range  of  potential  runs  from  addressing  the  A CO,  the  cogm/ant 
branch  chief  or  the  division  chief,  to  the  CAS  commander.  Under  appropriate 
circumstances,  contacts  with  contractor  management  are  appropriate  and 
productive  in  escalating  attention  to  contractual  issues  or  difficulties. 

Exemplifying  the  success  of  such  an  approach  is  the  Ogden  Air  l ogistics  Center 
(A1.C)  Procurement  and  Production  Directorate’s  constant  monitoring  of  contrac- 
tor delivery  schedules.  Such  monitoring  helps  to  reveal  trends  of  delinquency  that 
impact  on  the  Center's  ability  to  fill  field  requisitions  for  spare  parts.  The  fill  rate 
measures  support  of  weapon  system  readiness.  Tailored  letters  are  addressed  to 
appropriate  contractor  management  levels  when  delivery  schedules  are  not  met.  The 
result  is  success  in  focusing  corporate  management  attention  upon  contractual 
difficulties  that  affect  deliveries  for  contracts  now  delinquent  or  due  to  deliver  in  the 
near  term.  Trends  are  noted  in  the  letters,  and  the  impact  of  late  deliveries  upon  vital 
weapon  systems  is  highlighted.  Usually,  the  conscientious  contractor  will  identify 
the  causes  and  correct  the  delinquencies.  Copies  of  the  letters  are  sent  to  cogm/ant 
ACOs  to  maintain  continuity  of  contract  administration  and  A CO  awareness. 

There  are  also  other  opportunities  for  effective  use  of  written  communications  in 
contract  administration. 

Occasionally  the  customer’s  needs  may  change,  requiring  a contractual  modifica- 
tion to  reduce  quantities,  amend  shipping  instructions,  change  transportation,  or  to 
revise  delivery  schedules.  A termination  for  the  convenience  of  the  government  may 
even  be  required.  Such  actions  must  be  documented  in  the  contract  by  written 
communication. 

Another  avenue  of  written  communication  offering  answers  to  post-award 
problems  is  the  Defense  Materials  and  Priorities  System.  Authorized  by  the  Defense 
Production  Act  of  1950,  this  mechanism  directs  the  flow  of  materials  and  products 
to  the  nation's  military,  atomic  energv,  and  space  programs.  The  Defense  Priorities 
System  is  a rating  mechanism  for  establishing  precedence  on  orders  with  industry, 
e g..  DX  or  IX>,  indicating  the  importance  of  the  order.  The  precedence  is  fully 
extendable  throughout  the  industrial  chain  of  producers  and  may  be  advantageously 
incorporated  into  contracts  and  purchase  orders.  Orders  so  designated  take  prece- 
dence over  commercial  orders.  Using  the  Request  for  Special  Priorities  Assistance, 
DIB  Form  W,  materials  and  labor  scheduling  difficulties  can  be  reviewed.  Bringing 
into  play  the  powers  of  the  U.S.  Department  of  Commerce's  Domestic  and 
International  Business  Administration,  Bureau  of  Domestic  Commerce,  to  resolve 
production  difficulties  may  assure  delivery  of  required  supplies  While  this  proce- 
dure is  reserved  for  unusual  circumstances,  its  potential  is  formidable.  This  action 
may  be  initiated  by  contractors  or  the  government  buying  activity  after  routine 





34  | Defense  Systems  Management  Keueu 


efforts  fail  to  locate  sources  of  supply,  or  to  resolve  other  problems  such  as 
production  scheduling.  Often  the  suggestion  of  such  action  stimulates  initiative  and 
solutions  not  visible  earlier.  The  DIB  Form  W is  another  written  communication 
tool  available  for  use. 

Pnor  to  award,  during  formulation  of  the  contract,  a criticality  designator  (SCD) 
code  is  assigned.  Normally,  program  criticality  and  the  Defense  Priorities  System 
ratings  are  central  considerations  in  SCD  determination.  However,  after  award, 
should  contractor  performance  indicate  that  an  important  contract  is  being  ne- 
glected. causing  it  to  impact  other  contracts  as  hi  the  case  of  government  furnished 
property,  it  may  become  appropriate  to  upgrade  in  writing  the  SCD  code  from  "C" 
or  “B"  to  "A  " as  the  item  becomes  a limiting  or  pacing  item  in  a major 
procurement.  Some  other  means  to  provide  criticality  information  and  invoke  more 
intensive  contractor  surveillance  might  also  be  taken  in  the  form  of  a management 
inquiry.  These  actions  are  best  done  by  written  request.  As  a result  of  upgrading, 
closer  attention  can  be  expected  by  the  administering  DCAS  or  plant  representative. 

Following  sufficient  informal  coordination  and  exhortation  directed  toward  a 
deficient  contractor,  it  occasionally  becomes  necessary  to  serve  formal  written 
warning  in  the  form  of  a cure  notice,  a show-cause  notice,  or  a forebearance  notice. 
These  .letters  to  non-performing  contractors,  where  the  default  clause  exists,  can 
protect  the  rights  of  the  government  and  provide  solemn  warning  to  the  contractor. 
Deadlines  for  compliance  with  the  contract  terms  are  included.  The  prospect  of 
termination  action  is  raised.  Such  written  notification  suggests  the  possibility  of 
"divorce”  action,  should  appropriate  corrective  action  not  be  taken.  In  consequence, 
these  notices  warrant  careful  review  by  staff  legal  counsel  pnor  to  dispatch  by  the 
procuring  contracting  officer  (PCO). 

Use  of  the  ultimate  written  communicative  weapon,  termination  notices,  may 
become  necessary.  In  such  instances  the  needs  of  the  government  customer  may 
remain  unsatisfied.  Termination  action  may  be  a convenience  for  the  government, 
possibly  because  of  revised  requirements,  excusing  the  contractor  of  his  responsibil- 
ity. In  such  cases  the  government  may  be  liable  for  reasonably  incurred  costs.  In 
contrast,  termination  for  default  holds  the  contractor  pecuniarily  liable  for  excess  re- 
procurement costs  The  companions  to  termination  for  default  are  frequently  costly 
delays  in  receipt  of  the  needed  supplies  or  services  and  litigation  lasting  many 
months.  Consequently,  a termination  action  is  not  to  be  applied  casually. 

Each  of  the  foregoing  written  actions  or  tools  has  the  advantages  of  tangibility 
and  relative  permanence.  Carefully  conceived  correspondence  may  record  exactly 
what  is  intended  yet  the  cost  in  time,  human  energy,  and  equipment  resources  of 
written  documentation  should  be  weighed  against  the  advantages  to  be  realized 
when  selecting  a course  of  action. 
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Sometimes  personal  contact,  “ey eball-to-ey  eball."  adds  the  required  emphasis 
and  import  to  an  issue  of  concern  A better  understanding  and  greater  commitment 
may  result 

In  the  contract  administration  phase  before  and  after  awarding  a ptivuremeni 
action,  meetings,  both  external  to  the  government  and  internal,  max  become 
necessary  Personal  contacts  by  the  contract  administration  office  personnel  \x  ith  the 
contractor  at  the  plants,  both  prune  and  vendor,  may  be  priceless  expressions  of 
interest  and  sources  of  information  In  such  contacts  the  realities  of  contract 
performance  attain  a clarity  impossible  to  achieve  by  less  direct  contact  A well 
planned  visit  employing  reflective  listening  and  careful  observation  can  pay  hand 
some  rewards  in  terms  of  problem  anticipation,  detection,  and  resolution 

l he  benefits  of  such  plant  visits  are  seen  in  a broad  range  of  experiences  Foi 
example,  the  production  specialist  at  a IX'AS  office  v isited  a puxiucer  of  I 4 aircraft 
parts  prior  to  an  important  milestone  only  to  learn  that  the  contractor  had  failed  to 
place  a key  purchase  order  Also,  discussions  between  an  administrative  contracting 
officer  and  a plant  manager  surfaced  a recurring  contractor  production  control 
communications  roadblock  Fach  resulted  in  adequate  coi  rective  action 

On  another  ivcasion,  by  conducting  a pre-award  survey  m the  contractor's  plant, 
the  results  of  previous  “arm-chair"  surveys  were  expanded  to  accurately  reflect 
limitations  of  a prospective  contractor's  capabilities 

Further,  well-planned  conferences  between  buying  and  contract  administration 
activities  oiler,  go  a long  wax  toward  emphasizing  areas  of  concern  and  low  aid 
developing  genuine  personal  commitments  to  improvement  Such  activities  as 
annual  conferences  between  IX'AS  and  buying  activities,  program  reviews  with 
plant  representative  participation,  and  buying  office  visits  to  contract  adtnmistiation 
offices,  may  contribute  significantly  to  a stronger  dedication  to  the  achievement  of 
team  goals. 

Additionally,  face-to-face  visits  between  the  buying  office  and  the  contractoi 
merit  a brief  examination  Contacts  at  the  program  or  protect  manage!  level 
occasionally  fail  to  produce  the  desired  results  In  such  instances,  contact  with  the 
chief  operating  official  of  the  plant,  firm.  01  Kxiul  of  directors  max  become 
necessary  However,  meetings  between  the  1VO  and  the  corporate  contracts 
manager,  or  between  engineers  ot  other  functional  parties  can  also  be  quite 
productive  Written  memoranda  documenting  the  discussions  and  resulting  actions 
are  always  appropriate  Also,  an  awareness  of  the  provisions  of  the  contract  as  well 
as  the  nature  of  pitfalls  leading  to  constructive  changes  is  essential  to  the  government 
people  m contact  w uh  the  contractor 

Fxemphfymg  the  problem  of  constructive  changes,  a government  engtneet 
meeting  with  a contractor  engineer  max  remaik.  “\  ou're  designing  that  switch  ovei 
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here"  We've  always  had  it  over  there."  The  contractor's  implementation  of  such  a 
suggestion  would  likely  result  in  a claim  against  the  government  for  costs  incurred  to 
make  the  change.  Consequently,  representation  for  customer-contractor  discussions 
may  wisely  include  a contracting  officer  to  avoid  such  issues  w hich  could  add  cost, 
delay  delivery,  or  impact  performance  requirements  of  the  contract. 

When  multiple  contracts  are  involved  or  where  a complicated  contract  is 
underway,  periodic  internal  government  and/or  participating  contractor  review 
meetings  can  effectively  cover  a large  number  of  issues  at  a low  cost,  w hile  assuring  a 
successful  "partnership"  with  industry. 

Finally,  well-planned,  in-house  government  meetings  are  essential  to  strike  a 
unified  government  position  and  to  negotiate  differing  positions  or  to  direct  a 
common  position.  Such  mechanisms  as  management  attention  by  holding  meetings 
on  source  selection  procedures,  critical  item  management,  data  acceptance  proce- 
dures. and  Top  20,  10.  or  5 Problem  Rev  iews  are  to  Is  available  to  the  manager  on  a 
regular  or  ad  Aiv  basis.  Also,  holding  periodic  general,  internal  reviews  with 
accountable  persons  often  serves  to  direct  subordinates’  attention  to  key  areas  to  be 
discussed  and  resolved. 

In  a Nutshell 

l'he  opportunities  for  improved  understanding,  enhanced  commitment,  and 
problem  resolution  through  the  selective  application  of  various  communicative 
mechanisms  are  nearly  limitless.  Whether  the  telephone,  written  documents  or  face- 
to-face  meetings  are  used,  singly  or  in  combination,  communications  can  be  tailored 
by  the  program  manager  to  achieve  effective  contract  administration  resulting  in 
satisfaction  of  the  customer's  needs.  Careful  documentation  is  essential  to  preserve 
the  "corporate  memory"  and  may  play  a major  role  in  protests,  claims  or  litigation 
actions.  ^ et  prior  to  taking  action,  the  adv  antages  and  disadvantages  of  alternative 
courses  of  action  should  be  assessed.  II 
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THE  TWO-TIER  MATRIX 
ORGANIZATION  IN 
PROJECT  MANAGEMENT 

Dr.  William  C.  Wall,  Jr. 

The  matrix  organization  integrates  the  program  orientation  of  project  personnel 
with  the  speciality  orientation  of  functional  personnel.  In  its  original  form,  the 
matrix  is  a single-tier,  project-oriented  overlay  on  a functional  organizational 
structure.  Variations  of  the  original  form  have  resulted  in  a two-tier  matrix — a 
matrix  within  a matrix.  The  use  of  the  matrix  form  of  organization  in  military 
project  management  is  a familiar  practice,  but  the  two-tier  matrix  adds  a new 
dimension  to  the  concept.  In  large,  complex  organizations,  the  two-tier  matrix 
provides  for  greater  centralization  of  planning  and  control  through  definition  of  sub- 
projects.  It  substantially  assists  the  project  manager  in  accomplishing  program 
integration  by  encouraging  integration  at  the  sub-project  level. 

The  two-tier  matrix  is  a double-decker  in  the  project  structure — a matrix 
superimposed  on  a matrix.  It  is  a new  organization  form  prompted  by  the  continued 
growth  of  the  complexities  that  bred  the  original  single-tier  matrix.  The  two-tier 
matrix  is  a response  to  the  increasingly  complex  task  of  managing  military  projects. 

The  two-tier  matrix  is  not  an  unproven  concept.  For  example,  organization  has 
been  cited  as  a major  contributor  to  the  success  of  the  U.S.  Air  Force  F-15  fighter 
aircraft  acquisition  program.  The  management  structure  of  that  system  program 
office  was  a form  of  two-tier  matrix.1  The  U.S.  Army  HAWK  Project  Office  also  has 
been  recently  restructured  into  a two-tier  matrix  organization.1  Early  indications  are 
that  this  restructuring  has  improved  management  control  and  program  integration. 
In  both  cases,  implementation  of  the  concept  appears  to  have  facilitated  interaction 
among  participants. 

Single-Tier  Matrix  Design 

The  single-tier,  or  basic  matrix  organization  design,  evolved  during  the  1950s 
with  the  formal  development  of  military  project  management.1  The  rapid  growth  of 


'Gilbert  B.  Guarino,  Relva  L.  Lilly,  and  James  J.  Lindenfelser,  “Faith  Restored— The  F-15 
Program,"  Air  University  Review'll  (January-February  1976),  pp.  65-66. 

'U.S.,  Army  Missile  Command,  HA  WK  Project  Office  Management  System  Plan  (Redstone 
Arsenal,  Alabama:  HAWK  Project  Office,  14  October  1976),  pp.  1—2— 1—3. 

’An  excellent  comparative  description  of  functional,  project,  and  matrix  organizations  may  be  found 
in  Robert  Youker,  "Organization  Alternatives  for  Project  Managers,"  Management  Review  66  (No- 
vember 1977),  pp.  46-53. 
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Army  for  over  20  years.  Prior  to  his  current  assignment,  he  was  responsible  for  the  development, 
implementation,  and  operation  of the  SAFEGUA  RD  Management  Information  System.  He  has  served  as 
senior  management  advisor  on  numerous  government  ad  hoc  study  groups  and  committees.  Dr.  Wall 
holds  a BS  degree  in  mechanical  engineering  from  Lafayette  College,  an  M.  A in  public  administration, 
and  MBA  and  Ph  D degrees  from  the  University  of  Oklahoma. 
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technology  in  weapon  systems  since  World  War  II  and  a desire  to  minimize 
development  lead  time  have  resulted  in  an  increase  in  the  variety  of  information 
inputs  required  by  managers  from  the  scientific,  engineering,  and  administrative 
disciplines.  As  a consequence,  the  complexity  of  the  management  task  has  increased 
significantly  in  recent  years.  These  factors  gave  rise  to  the  concept  of  project 
management  and  to  the  single-tier  matrix  design. 

The  typical  military  project  office  is  established  to  accomplish  some  specific 
mission  or  objective  of  measurable  and  finite  duration.  In  this  sense,  a project  office 
differs  from  the  typical  functional  organization  that  assumes  a “going  concern” 
concept  of  enduring  mission.  Thus,  the  single-tier  matrix  layer  on  the  functional 
organization  is  intended  as  a temporary  overlay  on  the  base  functional  organization 
structure.  Typically,  the  project  office  staff  consists  of  a myriad  of  skills  or 
professions  in  an  interdisciplinary  array.  The  people  are  project  oriented,  devote  full 
work  time  to  the  project,  and  are  organizationally  assigned  to  the  project  office. 
These  people  work  directly  for  and  are  administratively  subordinate  to  the  project 
manager.  The  conventional  military  project  office  is  not  staffed  to  sufficient 
manpower  depth  to  be  self-supporting  and  must  draw  upon  manpower  from  outside 
the  office.  This  normally  results  in  the  acquisition  of  both  organic  and  inorganic 
support.  Organic  support  is  provided  by  functional  elements  integral  to  the  parent 
organization,  typically  a commodity  command.  Inorganic  support  involved  is 
acquired  from  other  government  agencies  and  the  private  sector. 

Figure  1 depicts  a typical  single-tier  matrix  consisting  of  six  functional  elements 
and  three  project  offices.  The  directors  of  the  functional  directorates  and  the  project 
manager  report  to  a single  individual — the  commodity  commander.  The  matrix 
design  below  these  levels,  however,  suggests  vertical,  horizontal,  and  diagonal 
relationships  among  participants.  It  also  suggests  that  many  functional  people 
operating  in  the  matrix  structure  have  two  group  memberships.  The  first  is  in  their 
speciality  or  functional  organization,  while  the  second  is  in  the  single  purpose  or 
project  environment.  The  single-tier  matrix  requires  that  these  people  serve  two 
masters — the  director  of  the  functional  directorate  and  the  project  manager.4  In 
other  words,  the  project  manager  and  the  functional  director  share  the  responsibility 
for  directing  the  efforts  of  these  individuals.  The  project  manager  exerts  project 
direction  while  the  functional  director  exerts  traditional  line  direction  over  the  same 
people  at  the  same  time.5 

The  project  management  office  structure  associated  with  a single-tier  matrix  is 
typically  function-oriented  and  is  illustrated  in  Figure  2,  which  depicts  six  functional 


'For  an  analysis  of  (he  two-boss  model  see  Stanley  M.  Davis  and  Paul  R Lawrence.  Matrix  (Reading. 
Massachusetts:  Addison-Wesley  Publishing  Company.  1977),  pp.  46-52. 

'Grover  Starling,  Managing  the  Public  SectortHomevood,  Illinois:  Dorsey  Press,  1977),  p.  193. 
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Figure  1 

SINGLE  - TIER  MATRIX 


Figure  2 

PROJECT  MANAGEMENT  OFFICE  IN  A SINGLE -TIER  MATRIX  STRUCTURE 
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elements.  Program  focus  is  achieved  through  the  natural  interaction  of  personnel 
within  the  project,  and  through  the  personal  drive,  initiative,  and  entrepreneurship 
of  the  project  manager.  It  is  he,  within  the  commodity  command,  who  directly 
assures  the  successful  production  of  goods  and  services.* 

Two-Tier  Matrix  Design 

The  two-tier  matrix  design  is  simple  in  concept.  It  employs  a matrix  design 
within  the  project  conceptually  similar  to  the  matrix  structure  employed  within  the 
commodity  command.  The  first  tier  of  the  two-tier  matrix  is  illustrated  in  Figure  3, 
depicting  six  functional  directorates  and  a single  project  office.  A comparison  of  this 
figure  with  Figure  I reveals  that  the  first  tier  of  a two-tier  matrix  for  a specific  project 
office  is  graphically  identical  to  the  single-tier  matrix  design  within  the  commodity 
command.  The  directors  of  the  functional  directorates  and  the  project  manager 
report  to  the  commodity  commander,  and  the  same  vertical,  horizontal,  and 
diagonal  relationships  among  participants  below  this  organizational  level  are  sug- 
gested. In  order  for  the  differences  to  begin  materializing,  however,  it  is  necessary  to 
look  at  the  second  tier  of  the  two-tier  matrix.  The  second  tier  is  illustrated  in  Figure 
4,  which  shows  six  functional  divisions  and  three  sub-project  elements. 


Figure  3 

FIRST  TIER  OF  TWO  TIER  MATRIX 


*rtm  thesis  is  developed  in  Marshall  Dimock.  'Revitalized  Program  Management."  IWr  .4, /m/m* 
muon  Kenr»  lS(May-June  l>»7g>  |<W  XV* 
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Comparison  of  Figure  4 with  Figure  2 results  in  the  immediate  observation  that 
the  project  management  office  organizational  structure  in  a two-tier  matrix  is 
markedly  different  from  the  project  management  office  in  a conventional  or  single- 
tier matrix.  Specifically,  in  the  two-tier  matrix,  the  project  management  office  has  an 
internal,  horizontal  focus  not  evident  in  the  single-tier  matrix.  It  is  this  horizontal 
orientation  that  materially  assists  the  project  manager  in  his  role  as  program 
integrator.  It  is  this  horizontal  or  sub-project  dimension  that  also  differentiates  the 
two-tier  from  the  typical  single-tier  matrix.  This  innovation  adds  versatility  to 
conventional  matrix  management  while  creating  constructive,  yet  challenging, 
management  interactions  among  the  participants. 

The  second  tier  of  the  matrix  is  depicted  in  context  with  the  first  tier  in  Figure  5, 
where  the  complete  two-tier  matrix  is  illustrated.  It  may  be  seen  that  the  two-tier 
matrix  is  not  just  another  routine  way  of  organizing  a project  management  office.  It 
is  a significant  departure  from  traditional  designs  and  adds  a new  dimension  to 
project  management. 

Two-Tier  Matrix  Management  in  Operation 

The  hierarchic  or  chain  of  command,  the  project,  and  the  functional  lines  of 
authority  in  two-tier  matrix  management  are  highly  complex.  Figure  6 graphically 


Figure  4 

SECOND  TIER  OF  TWO -TIER  MATRIX 
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Figure  5 

TWO  TIER  MATRIX 


displays  these  relationships  from  the  commodity  commander  down  through  the 
branch  chief  level — a total  of  four  discrete  levels  of  typical  chain  of  command.  A 
careful,  point-by-point  discussion  of  each  of  the  participants'  roles  in  the  two-tier 
matrix  structure  will  provide  insight  into  these  relationships. 

It  should  be  recognized  that  although  the  commodity  commander  supervises  the 
project  manager  and  the  functional  director,  both  of  whom  are  in  the  matrix,  he 
himself  is  not  a member  of  the  matrix.7  It  is  his  responsibility  to  insure  that  both 
individuals  perform  their  respective  missions  successfully  and  to  adjudicate  differ- 
ences between  them.  The  commodity  commander  has  a corporate  outlook.  He  does 
not  share  his  power  with  anyone  and  he  is  solely  responsible  for  the  aggregate  of 
weapon  system  programs  comprising  his  commodity  command  mission.  The  com- 
modity commander’s  position  is  a unique  blend  of  project  and  functional  responsibil- 
ity. He  facilitates  the  integration  of  the  individualized,  inward  focus  of  project 
managers  with  the  universalized,  outward  focus  of  functional  directors. 

The  project  manager  depicted  in  Figure  6 is  responsible  for  a single  project  made 
up  of  multiple  sub-projects.  For  the  sake  of  simplicity,  only  one  sub-project  manager 
and  one  functional  division  chief  reporting  to  the  project  manager  are  illustrated. 


Davis  and  Lawrence,  Matrix,  p 47 
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CONCEPTUAL  MODEL  OF  TWO  TIER  MATRIX  MANAGEMENT 
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The  project  manager  is  immediately  subordinate  to  the  commodity  commander  and 
unifies  project  affairs.  He  is  at  the  same  hierarchic  level  as  the  functional  director 
organizationally,  but  he  is  authorized  to  control  the  activities  of  the  functional 
director  as  they  relate  to  his  assigned  project  and  within  prescribed  limits.'  In  other 
words,  he  has  the  authority  to  issue  project  direction  to  the  functional  director. 

The  functional  director  illustrated  in  Figure  6 is  responsible  for  a total  functional 
speciality  such  as  procurement,  product  assurance,  supply  or  maintenance.  He,  like 
the  project  manager,  is  immediately  subordinate  to  the  commodity  commander. 
Unlike  the  project  manager,  however,  his  is  a multi-boss  position.  He  receives 
command  direction  from  the  commodity  commander  and  project  direction  from  all 
of  the  project  managers  assigned  to  the  commodity  command.  The  functional 
director  provides  functional  facilitation  to  all  projects  and  his  functional  authority 
may  occasionally  conflict  with  project  goals.  As  Figure  6 illustrates,  this  functional 
feedback  is  generally  directed  to  the  next  lower  hierarchic  level  in  the  project  office 
organization.’ 

To  this  level  in  the  chain  of  command,  the  interrelationships  do  not  differ  from 
those  of  the  single-tier  matrix  design.  It  is  at  the  next  level— that  of  the  division 
chiefs — that  the  differences  between  single-tier  and  two-tier  matrix  interrelation- 
ships begin  to  emerge. 

The  sub-project  manager  within  the  project  office  is  responsible  for  a discrete, 
separately  identifiable  element  of  the  total  program.  As  an  example,  one  sub-project 
manager  might  be  responsible  for  all  foreign  military  sales  of  the  weapon  system, 
another  for  international  co-production  and  technology  transfer,  and  another  for  the 
U.S.  Army  program.  This  is  essentially  the  breakout  used  in  the  HAWK  Project 
Office."  In  the  case  of  the  F-15  program,  the  areas  of  responsibility  were  associated 
with  major  segments  of  the  development  program  such  as  the  airframe,  engine, 
armament,  and  training." 

Some  of  the  principal  criteria  affecting  establishment  of  sub-project  managers  are 
the  degree  to  which  program  differentiation  is  required  and  attainable,  the  criticality 
of  resource  control,  and  the  significance  of  the  sub-project.  The  sub-project  manag- 
ers may  operate  as  one-person  offices  or  have  small  groups  of  high  level  personnel 

'As  an  example,  a project  manager  may  direct  a contracting  officer  to  execute  a contract  for  certain 
goods  or  services  He  may  not,  however,  dictate  the  type  of  contract  since  this  determination  is  within  the 
contracting  officer's  area  of  responsibility  and  authority. 

*The  feedback  is  not  unlike  the  example  suggested  in  the  preceeding  footnote  It  also  applies  to  fiscal 
resources  in  fund  appropriations  not  managed  directly  by  the  project  manager  In  other  words,  the  protect 
manager  may  not  he  provided  all  the  funds  he  requires  by  a functional  director  In  this  example,  the 
feedback  would  be  from  the  functional  director  to  the  project  office  divtaon  chief 

“II  S.,  Army  Missile  Command.  HA  WK  Pmtect  Office  Management  System  Plan,  p I x 

"(iuartno,  I illy,  and  I indenfelser,  p 65 
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assigned  to  them  to  assist  in  carrying  out  their  sub-project  program  responsibility. 
The  creation  of  sub-projects  focuses  added  intensified  management  within  the 
project  office.  As  indicated  in  Figure  6,  the  sub-project  manager  receives  command 
direction  from  the  project  manager  and  issues  sub-project  direction  to  the  functional 
division  chiefs  within  the  project  office.  As  a practical  matter,  the  relationship 
between  the  sub-project  manager  and  the  functional  division  chiefs  in  the  project 
office  is  strikingly  similar  to  that  between  the  project  manager  and  the  functional 
directors. 

The  functional  division  chiefs  in  the  project  office  are  key  individuals  in  the  two- 
tier  matrix.  The  ultimate  success  or  failure  of  the  concept  rests  largely  on  their 
shoulders.  As  indicated  in  Figure  6,  the  project  office  functional  division  chief 
positions  are  multi-boss  positions.  The  project  office  functional  division  chiefs  are 
subordinate  to  and  receive  command  direction  from  the  project  manager,  sub- 
project  direction  from  the  sub-project  manager,  functional  feedback  from  the 
functional  director,  and  issue  project  direction  to  the  functional  directorate  division 
chiefs  and  functional  direction  feedback  to  the  sub-project  manager.  The  project 
office  functional  division  chiefs  are  the  focal  points  for  information  flow.  They  are 
the  true  integrators  of  the  single-purpose  special  requirements  of  the  project  with  the 
multi-purpose  general  responsibilities  of  the  functional  directorates.  It  is  their  duty 
to  optimize  the  use  of  project  resources,  serve  as  interface  between  sub-project 
managers  and  functional  directorate  division  chiefs,  and  minimize  project  uncertain- 
ties. They  sit  in  the  catbird's  seat  of  project  activity.  Figure  b also  illustrates  the  next 
lower  level  in  the  hierarchy,  and  the  interrelationships  at  this  level  are  conceptually 
identical  to  those  just  described  for  the  division  chief  level. 

Implications  of  the  Two- Tier  Matrix 

The  matrix  organization  is  considered  an  organization  form  of  the  future.".  The 
two-tier  matrix  is  visible  proof  that  project  management  of  complex  weapon  systems 
is  conceptually  alive  and  well.  It  helps  establish  an  organization  structure  with 
greater  flexibility  for  responding  to  external  pressures  and  influences.  It  creates 
interrelationships  among  participants  that  stimulate  group  initiative  and  enhance 
the  group's  ability  to  cope  with  the  complexities  of  typical  weapon  system  projects. 
This  same  mechanism  tends  to  decentralize  routine  decision  making  by  moving  it 
closer  to  the  points  of  action. 

Many  of  the  typical  pathologies  associated  with  the  single-tier  matrix  design  arc 
equally  evident  in  the  two-tier  matrix.".  On  the  other  hand,  the  two-tier  matrix 


'Starling,  pp  l"*l  I'M 
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forms  the  foundation  for  even  greater  utility  in  the  project  management  concept.  It 
extrapolates  the  single-purpose  emphasis  of  the  original  matrix  design  without 
duplicating  the  functional  capability  contained  in  the  project  office.  By  providing  a 
vehicle  for  concentrating  planning  and  control  at  the  sub-project  level,  the  two-tier 
matrix  encourages  the  grouping  of  similar  program  elements  within  the  project.  This 
aspect  helps  bring  greater  focus  to  major  program  elements  of  multifaceted  projects 
and  insures  that  desired  management  emphasis  is  afforded  each. 

The  two-tier  matrix  design  is  not  a panacea  for  p»v>r  or  inept  management.  In 
fact,  proper  implementation  requires  extraordinary  skill  on  the  part  of  all  concerned 
Matrix  managers  face  many  challenges  in  the  proper  accomplishment  of  then 
assigned  responsibilities,  but  the  two-tier  matrix  design  is  a solid  means  for 
successfully  conducting  complex  and  urgent  programs.  Intensive  management  of 
selected  weapon  system  programs  remains  essential.  The  two-tier  matrix  design  is  in 
harmony  with  this  compelling  need  and  underscores  the  continuing  search  for  order 
in  the  management  of  large-scale,  complex  endeavors.  || 


MANAGEMENT  UPDATE: 
THE  ARMY  SCIENCE  AND 
TECHNOLOGY  PROGRAM 

Dr.  Marvin  E.  Lasser 

The  Army  has  relied  for  a long  lime  on  its  science  and  technology  program  to 
provide  a technological  edge  over  its  potential  adversaries.  As  a result,  the  U S 
Army  will  soon  have  the  best  tanks,  helicopters,  and  anti-tank  missiles  in  the  world. 
However,  there  are  capability  areas  where  we  do  not  fare  as  well.  For  example,  our 
potential  enemies  out-range  us  in  artillery  and  have  more  capable  air  defense  gun 
systems.  We  need  to  improve  our  technological  capabilities  so  that  we  can  achieve 
superiority  across  the  board.  We  cannot  settle  for  less. 

We  must  insure  that  we  get  maximum  return  within  the  available  funding  levels. 
One  way  to  do  this  is  to  place  our  trust  in  the  people  responsible  for  managing  the 
Army's  science  and  technology  programs.  Our  systematic  efforts  to  do  this  have 
resulted  in  a major  decentralization  of  the  management  of  Army  long-term  technol- 
ogy efforts. 

The  first  step  in  decentralization  was  to  provide  the  responsible  Army  laboratory 
director  with  a block  of  funds  for  a given  fiscal  year.  The  process  for  doing  this  is 
known  as  Single  Program  Flenient  Funding  (SPEF).  Under  SPFF.  the  laboratory 
director  proposes  a set  of  prioritized  tasks  and  indicates  what  he  expects  to 
accomplish  during  the  coming  year.  His  approved  program  is  then  “block"  funded 
and  the  director  is  held  responsible  for  the  performance  of  his  laboratory.  In  this 
concept,  the  laboratory  director  is  free  to  change  his  program  at  any  time  to  take 
advantage  of  technological  opportunities.  The  budget  for  each  laboratory  is  strongly 
influenced  by  accomplishments  in  the  previous  year's  program  and  the  content  of  the 
proposed  program 

Before  SPF.F.  the  laboratory  director  had  considerably  less  control  over  what 
was  done  in  his  laboratory  He  had  little  freedom  to  change  projects  since  each  work 
unit  was  funded  individually.  There  were  technical  specialists  on  the  Army  staff  who 
controlled  the  funds  in  their  particular  technical  area.  Washington-based  specialists 
tn  propulsion,  physics,  meteorology,  social  sciences,  medical  services,  etc  . not  only 
pushed  their  areas  of  expertise  in  competition  with  each  other,  but  also  controlled 
what  was  done  in  the  laboratories  in  their  areas  of  expertise  l aboratory  directors 
were  justifiably  frustrated  by  this  process  m which  they  could  not  even  move  funds 
from  the  weaker  programs  to  the  potentially  more  promising  ones  without  consider- 
able administrative  effort. 

Delegation  of  authority  to  the  laboratory  director  was  often  agreexl  to  in  principle 
but  not  readily  adopted  for  several  reasons  directly  related  to  imagined  Congression- 
al objections.  One  reason  put  forth  was  that  if  laboratory  funds  were  combined  into 
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two  program  elements  per  laboratory,  one  in  research  (6.1)  and  one  in  exploratory 
development  (6.2),  then  Congress  would  see  them  as  new  large  programs  and  might 
make  cuts,  not  recognizing  a consolidation  of  many  programs.  Experience  has 
shown  this  not  to  be  the  case.  Congress  has  never  cut  a SPEF  program  because  of  its 
size.  In  fact,  the  biggest  SPEF  programs  have  fared  very  well  in  Congress.  (Note: 
Since  the  Army  6.1  program  consists  of  only  two  program  elements,  which  are 
further  divided  into  projects,  the  block  funding  for  6. 1 is  often  referred  to  as  Single 
Project  Funding,  or  SPF.) 

A second  reason  theorized  was  that  Congress  would  object  to  what  many 
thought  would  be  a lack  of  visibility  of  program  content.  Again,  this  was  not  the 
case.  By  using  SPEF,  the  laboratory  director  was  better  able  than  before  to  explain 
his  program.  He  could  now  describe  his  program  as  a total  entity  and  could  readily 
portray  how  the  various  parts  fit  together.  Advanced  planning  for  SPEF  was  well 
coordinated  with  members  of  Congress  and  their  staffs,  and  they  gave  the  Army 
their  full  support. 

Although  the  move  to  delegate  authority  via  SPEF  was  a most  important  one,  it 
was  only  one  of  a number  of  changes  that  were  made  to  improve  the  R&D  process. 

The  Problem 

Clearly,  the  lab  directors  were  happy  to  get  the  authority  that  went  with  SPEF. 
However,  with  authority  came  responsibility  and  the  question,  "What  are  the 
priority  requirements  my  laboratory  should  address?"  The  answer  to  this  question  is 
provided  by  the  Training  and  Doctrine  Command  (TR  ADOC),  which  speaks  for  the 
“user."  When  the  problem  materialized,  the  TRADOC  organization  was  not  geared 
to  interface  with  the  Army’s  science  and  technology  program.  Means  had  to  be 
developed  to  bring  the  user  and  developer  together  early  in  the  developmental 
process.  It  was  necessary  to  encourage  the  user  to  articulate  his  needs  in  such  a way 
that  the  laboratories  could  understand  and  respond. 

The  Solution 

The  vehicle  developed  to  identify  user  requirements  was  the  Science  and 
Technology  Objectives  Guide,  or  STOG.  The  STOG  is  divided  into  capability 
categories  and  sub-categories  as  shown  in  Figure  1 . Each  major  capability  category 
section  is  a listing,  in  priority  order,  of  the  user's  needs  within  that  category.  For 
example,  in  the  capability  category  "other  combat  support."  the  highest  priority 
need  is  rapidly  emplaced  minefields  with  specific  characteristics  such  as  variable 
time  activation,  remote  deactivation,  and  firing  based  on  discrimination  sensors  The 
next  priority  Science  and  Technology  Objective  (STO)  addresses  improved 
equipment/techniques  for  detection  and  neutralization  of  minefields  Going  further, 


Li 


ATEGORIES  (CAPCATS) 


z 

o 

u 


S < 


K < 

'£  u 


A _ ~ 


g i 


E y 

ll| 

<5  I 

i if 

55?. 


i>i  2 


g p 


i 3 J< 


Management  Update  | 49 


2 

* 


I 


2P  — 

SA 


a i i S > V ? 
.2  5 5 5 ,*  5 < 


I 

is 


5-  ^ 


s -g 

^5 


gig  oca 

= ; J = t , 

' I c ' " £ 


* 

§ 


35 

II 


3 

A 


E 

* 


M 


E§  2 


z 

o 

u 


5 S 3 c s 

|i-i 

r * g g 2 t 
g % ! j $ % 

S'  *-  vJ  < CL  O 


? S 

|7  , I- 

5 3 5 it 
|£ 

l|4  | 

=j.  i j .3  3 *' 

If  S 

i< 


s 

-2  5 


^ * U% 
5 a J 


- 5 I 
V 5 2 

. !<  3 


14 


a?  1 1 g t 

l|SlsP 

/ ,/\  uj  i fv  t 


*2 

'll 


5 ^ 


I ill 

5 3 £ ^ 


ijs 
It  1 
III 

I j!< 


1 

1 


I 

ji 


5 

a * 
? f 

15 


||l 

HI 


£ 

i 


2 


|jf*| 


5// 


I 


i 

* 


SO  I Defense  Systems  Management  Review 


the  next  one  describes  airdrop  system  requirements.  An  important  point  to  note  is 
that  these  STOs  are  clearly  user-oriented  rather  than  developer-oriented.  Each  STO 
lists  the  user  proponent  most  concerned  with  the  objectives  delineated,  as  well  as  the 
laboratory  assigned  primary  responsibility  to  see  that  the  STO  is  adequately 
addressed. 

Each  capability  category  also  contains  a background  section,  a discussion  of  the 
general  capabilities  required  (e  g.,  rapid  enhancement  of  mobility  for  friendly  forces 
and  a counter-mobility  capability  to  impede  enemy  forces),  and  a concept  of 
operations. 

The  purpose  of  the  concept  of  operations  portion  is  to  provide  the  laboratory 
director  with  the  “big  picture”  of  what  is  to  be  accomplished.  He  can  then  better 
understand  the  prioritization  of  the  STOs.  which  provides  a basis  for  meaningful  and 
innovative  management  as  requirements  change  or  as  research  opportunities  become 
evident 

It  is  important  to  reiterate  that  the  STOG  is  a requirements  document  reflecting 
the  needs  of  the  user.  Notwithstanding  this  basic  precept,  the  user  community  had  to 
have  help  in  preparing  the  document.  One  difficulty  was  that  the  TRADOC  staff 
sometimes  felt  they  could  not  properly  reflect  their  future  needs  in  a way  the 
laboratory  scientist  could  understand.  It  was  a classic  communications  gap  with  the 
developer  anxious  to  get  requirements  but  not  sure  how  to  go  about  getting  them, 
and  the  user  holding  the  opinion  (mistakenly)  that  there  was  no  easy  way  to 
communicate  needs  in  a way  laboratory  scientists  would  understand  and  be  able  to 
respond. 

Another  problem  was  the  inherent  difficulty  of  adequately  defining  require- 
ments. Often  the  requirement  cannot  even  be  formulated  without  a great  deal  of 
communication  between  user  and  developer.  Also,  the  requirements  frequently 
change  (or  mature)  during  the  course  of  effective  user/developer  communications. 

Once  started,  the  dialogue  between  the  scientist  and  the  user  ran  very  smoothly. 
The  first  draft  version  of  the  STOG  was  prepared  by  the  staffs  of  the  Director  of 
Army  Research  and  the  Deputy  Chief  of  Staff  for  Operations  and  Plans  (DCSOPS), 
but  it  was  based  on  a draft  of  requirements  that  had  been  established  in  one  form  or 
another  within  the  TRADOC  community.  It  only  needed  a start. 

The  second  iteration  of  the  STOG.  called  STOG-78.  was  published  in  April  1V77 
and  the  third  iteration.  STOG-79,  will  be  active  until  the  next  update  in  April  lt>7d. 
Laboratory  directors  have  used  STOG  to  redirect  their  programs  to  the  Army’s 
highest  priority  needs,  and  management  program  reviews  have  used  it  as  a basis  to 
judge  the  responsiveness  of  laboratory  programs  to  the  Army's  needs.  Joint  labora- 
tory program  reviews  are  held  annually  in  the  spring  and  it  is  noteworthy  that  the 
TRADOC  representative  is  a key  figure.  He  compares  programs  against  the  stated 
needs.  This  has  a significant  impact  on  program  planning  for  the  future.  The  STOG 
is  constantly  improved  and  refined  in  subsequent  editions. 
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Caution 

A caution  which  must  be  kept  in  mind  in  any  discussion  of  requirements 
documents  related  to  a science  and  technology  base  program  is  that  if  we  try  to 
define  the  goals  for  the  entire  science  and  technology  program  we  are  making  a 
serious  mistake.  To  do  so  would  prove  to  be  too  constraining  for  a meaningful, 
productive  program.  We  must  leave  room  for  innovation.  The  Army  is  sensitive  to 
this  concern.  To  quote  from  the  executive  summary  of  the  STOG:  “Nothing  in  this 
STOG  is  intended  to  depart  from  the  essential  SPF/SPEF  management  concept  that 
R&D  directors  must  determine  where  they  can  make  the  most  significant  science 
and  technology  contributions  to  the  known  or  presently  unforseeable  needs  of  the 
Army." 

It  has  been  estimated  that  approximately  70  percent  of  the  6.2  exploration 
development  programs  should  be  in  response  to  STOG  objectives.  The  other  .10 
percent  or  so  represents  funds  available  for  the  laboratory  director  to  use  to  pursue 
technological  opportunities  as  they  arise.  Many  new  technological  opportunities  can 
be  forseen  by  the  laboratories  as  the  SPEF  plans  are  being  prepared,  and  can  be 
included  in  the  plan.  When  opportunities  for  innovation  become  apparent  after  the 
annual  plan  is  published,  the  director  still  has  the  freedom  to  readjust  his  program. 
At  the  end  of  the  year  he  and  his  lab  are  judged  on  the  efficacy  of  the  overall 
program.  The  director  does  not  have  to  adhere  entirely  to  the  plan  he  laid  out  at  the 
beginning  of  the  year;  on  the  contrary,  he  can  make  meaningful  changes.  But  he  is 
also  expected  to  address,  wherever  feasible,  the  highest  priority  needs  that  his 
laboratory  can  satisfy. 

Management  Review 

Given  the  STOG  and  given  the  laboratory  programs,  management  now  has  the 
requisite  information  to  evaluate  return  on  the  science  and  technology  investment. 
This  is  done  by  the  Research  Development  and  Acquisition  Committee  (RDAC)  in 
preparation  for  next  year’s  budget.  Each  proposed  or  current  program  is  examined 
to  determine: 

• What  assumptions  lead  us  to  work  in  the  sub-capability  category; 

• What  the  major  thrusts  of  the  programs  are; 

• What  the  STOG  calls  for  within  the  sub-capability  category; 

• What  pacing  problems  the  laboratories  see; 

• The  work  being  done  to  solve  each  pacing  problem; 

• The  laboratories  doing  the  work,  and; 

• The  dollar  amount  being  spent  on  each  problem  area. 
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By  reviewing  by  capability  category  rather  than  by  laboratory  organization,  the 
Army's  total  program  in  a given  functional  area  is  readily  identifiable. 

Review  by  capability  categories  provides  an  excellent  method  for  overview  of 
that  portion  of  the  Army’s  science  and  technology  program  oriented  towards  solving 
problems  stated  in  the  STOG.  It  turned  out  (notwithstanding  the  guidance  that  a 
reasonable  percentage  of  the  program  should  be  independent  of  STOG  require- 
ments) that  over  90  percent  of  the  6.2  program  submitted  could  be  correlated  with 
the  STOG.  It  is  not  clear  how  mucn  of  this  is  a true  reflection  of  the  program  and 
how  much  is  a “forced"  correlation  to  show  relevance.  The  percentage  was,  of 
course,  much  smaller  for  the  6. 1 program. 

Careful  study  of  the  laboratory  inputs  showed  few  examples  where  the  stated 
correlation  with  the  STOG  was  unrealistic.  This  may  well  indicate  that  too  much  of 
the  current  6.2  program  is  closely  coupled  to  readily  forseeable  application.  Further 
study  of  this  aspect  clearly  is  required. 

Research,  Development,  and  Acquisition  Committee 

The  final  step  in  the  management  process  was  the  adjustment  of  funds  based  on 
the  information  organized  by  capability  category.  Balancing  and  readjusting  R&D 
and  procurement  funds  is  normally  carried  out  by  a group  known  as  the  Research, 
Development , and  Acquisition  Committee  (RDAC).  This  committee  has  representa- 
tion from  the  R&D  and  operations  communities,  from  TRADOC  representing  the 
user,  as  well  as  representation  from  the  development  organizations. 

This  year  the  RDAC  added  a special  session  that  addressed  the  science  and 
technology  base  exclusively.  The  review  by  capability  categories  provided  the 
visibility  and  understanding  of  the  program  content,  and  this  in  turn  enabled  the 
RDAC  group  to  make  funding  adjustments  within  the  6. 1 and  6.2  program  in  a more 
meaningful  and  appropriate  way. 

It  was  clear  to  those  participating  in  the  RDAC  process  that  the  science  and 
technology  base  is  highly  responsive  to  the  needs  of  the  Army.  These  programs  can 
compete  favorably  for  funds  when  compared  either  to  major  systems  or  to  other 
systems  that  are  well  along  in  development. 

Conclusion 

The  Army  has  developed  techniques  that  have  provided  improvements  in  the 
management  of  its  science  and  technology  programs,  based  on  the  premise  that  we 
put  the  authority  and  responsibility  for  laboratory  programs  where  it  belongs— with 
the  laboratory  director. 
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If  this  authority  is  truly  delegated  to  the  director,  he  cannot  then  he  told  how  to 
run  his  programs.  Obviously,  however,  this  does  not  mean  that  he  cannot  be  helped 
with  advice  on  what  the  Army  needs. 

This  delegation  of  authority  cannot  be  considered  a blank  check.  It  carries  with  it 
the  responsibility  to  provide  the  Army  a return  on  its  investment,  either  from  an 
individual  laboratory  or  from  a number  of  laboratories  working  in  concert  Hy 
compiling  the  major  program  thrusts  by  capability  categories  it  is  possible  to 
determine  which  areas  are  not  adequately  covered  and  therefore  where  additional 
emphasis  is  needed  l'his  same  display  of  program  content  clearly  portrays  the 
importance  and  relevance  of  the  on-going  programs,  but  when  changes  in  program 
emphasis  are  required  (found  to  be  the  case  in  only  a small  fraction  of  the  total 
program)  this  can  be  done  in  a knowledgeable  way.  I 
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The  quality  assurance  (QA)  discipline  has  long  been  recognized  as  an  effective 
tool  for  management  in  producing  hardware  systems.  Only  recently,  however,  have 
people  asked  the  question,  "Why  not  software,  too?"  In  the  late  fifties,  the  military 
issued  a QA  specification  that  is  the  basis  of  almost  every  quality  program  used  by 
contractors  doing  business  with  the  government  today  This  specification.  M1L-Q- 
9858A.  Quality  Program  Requirements,  was  intended  to  cover  all  supplies  and 
services  when  it  was  referenced  in  an  item  specification,  contract,  or  order.  However, 
the  government  now  is  viewing  software  as  a separate  deliverable  item  and,  hence, 
subject  to  the  requirements  of  the  QA  specification.  This  has  presented  something  of 
a problem  both  to  the  contractors  and  to  the  Defense  Contract  Administration 
Service  (DCAS),  which  is  the  government’s  QA  organization.  No  one  really  knows 
how  to  apply  MIL-Q-9858A  as  a software  specification. 

Twenty  years  of  hardware-oriented  implementation  of  the  QA  specification  has 
created  a tremendous  inertia.  Organizations  have  been  built  and  people  have  spent 
their  entire  careers  studying  MIL-Q-9858A  as  it  relates  to  a single  step  in  a 
manufacturing  process.  A separate  and  distinct  professional  discipline  has  grown  up 
within  this  environment.  The  government  QA  function  also  matured  during  this 
time.  A cross-pollination  process  in  the  late  fifties  and  sixties  involving  both 
government  and  industry  personnel,  in  combination  with  the  DCAS  creation  of  the 
handbook  for  evaluating  a contractor's  quality  program,  created  a “quality" 
community. 

This  community  grew  and  prospered  and  began  to  speak  a common  language. 
The  quality  programs  across  industry  took  on  a striking  similarity,  all  based  on  a 
common  understanding  and  approach  to  MIL-Q-9858A.  This  community  did  not 
address  software  because  software  was  not  a part  of  the  problem.  No  one  saw  the 
trend  that  was  developing.  As  hardware  systems  grew  more  powerful,  so  did  the 
software  systems  that  operated  them.  As  program  performance  requirements  grew, 
the  size  and  complexity  of  programs  also  grew.  This  growth  dictated  more  memory 
and  processor  speed.  As  the  engineers  responded  with  new  generations  of  hardware, 
the  programmers  used  it  up  and  asked  for  more.  There  is  a significant  point  to  be 
made  here.  New  and  more  powerful  hardware  generations  have  been  born  out  of 
improved  technology  and  engineering  methodology  Programming,  on  the  other 
hand,  has  not  benefitted  from  any  quantum  strides  in  tools,  techniques  and 
methodologies.  Then,  as  now,  it  was  a laborious,  intensive,  artful  endeavor.  In  spite 
of  some  innovations  such  as  high-level  languages,  structured  programming  and  the 
like,  programming  productivity  has  not  nearly  kept  pace  with  the  demand. 


H M Knight  i\  Quality  Serna's  Manager  at  IBM's  Manassas.  I a.  facility  He  is  rrspiyn\ible  for 
defining  and  implementing  a software  quality  assurance  png  ram  there  and  extending  the  quality  concept 
to  include  reliability  Mr  Knight  holds  a H S I I degree  fmm  the  l nnersity  of  Honda 
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Further,  we  have  seen  a change  in  the  cost  ratio.  Tremendous  gains  have  been 
made  in  performance  of  digital  hardware  systems  over  the  past  twenty-five  years. 
Hardware  costs  have  gone  down  dramatically  because  the  labor  required  to  produce 
the  new,  higher  performance  hardware  is  no  more  and  often  less  than  that  required 
to  manufacture  the  old,  lower  performance  systems.  A commercial  job  mix  of  about 
1,700  operations  cost  approximately  $14.50  to  run  in  1955.  Today,  the  same  job  costs 
approximately  twenty  cents.  Moreover,  the  run  time  on  this  job  has  been  reduced  by 
a factor  of  70.'  The  indications  are  that  the  hardware  cost-to-performance  ratios  will 
continue  to  improve.  Because  of  this,  the  Department  of  Defense,  for  one,  is  finding 
software  to  be  by  far  the  most  expensive  item  in  the  inventory.  At  the  spring  1977 
meeting  of  the  National  Security  Industrial  Association  Quality  and  Reliability 
Committee  it  was  reported  that  DOD  is  spending  more  than  $5  billion  annually  on 
specialized  software  alone.'  This  is  considered  a conservative  estimate  and  does  not 
include  general-purpose  automatic  data  processing  software.  A hardware-to- 
software  breakdown  of  the  total  materiel  system  cost  to  DOD  since  1955  shows  an 
astonishing  trend.  Software  cost  as  a percentage  of  total  materiel  system  costs  has 
steadily  risen  from  less  than  20  percent  in  1955  to  more  than  80  percent  today,  and  it 
is  still  increasing. 

There  are  two  basic  reasons  for  this: 

• 5/>e— On  the  average,  software  systems  today  are  an  order  of  magnitude  larger 
than  they  were  20  years  ago.  Memory  has  become  cheap.  An  adaptation  of 
Parkinson’s  law  says  that  the  number  of  program  instructions  will  increase  to  the 
limit  (and  often  beyond)  of  available  storage.  Therefore,  the  tendency  is  to  keep 
adding  function  until  the  core  is  gone. 

• Complexity— Not  only  have  the  software  systems  been  packed  with  function,  the 
functions  and  functional  interfaces  are  increasingly  more  complex.  The  nature  of 
the  problems  we  are  trying  to  solve  with  computer  programs  today  did  not  exist 
only  a few  years  ago.  Consider  the  problem  of  detecting,  classifying,  and 
successfully  defending  against  a large-scale  1CBM  attack.  That  was  the  problem 
faced  by  the  system  designers  on  an  antiballistic  missile  program.  They  re- 
sponded with  one  of  the  largest,  most  complex,  most  expensive  systems  ever 
developed.  Consider  the  Navy’s  problem  in  dealing  with  the  super  submarines  of 
today.  Consider  the  complexity  of  today's  business  systems  against  20  years  ago 
or  just  10  years  ago.  There  may  have  been  a time  when  there  was  an  alternative  to 
automatic  data  processing  to  keep  our  way  of  life  whole,  but  not  anymore 
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The  demand  for  more,  larger,  and  more  complex  programming  is  real  and 
inescapable.  Therefore,  we  must  accept  size  and  complexity  as  a fact  of  life,  a part  of 
the  nature  of  our  society,  and  learn  to  deal  with  it  in  a more  systematic  fashion. 
Today,  it  seems  the  only  way  to  get  more  programming  is  to  add  more  people,  but 
more  people  cost  disproportionately  more  money. 

An  interesting  observation  is  that  one  reason  programmable  machines  were 
invented  in  the  first  place  was  that  an  efficient  and  inexpensive  way  was  needed  to 
solve  a variety  of  problems  without  changing  hardware.  It  seems  now  we  need  an 
efficient  and  inexpensive  way  to  solve  a variety  of  problems  without  changing  the 
software,  because  making  those  changes  has  become  a very  expensive  proposition. 
Since  we  do  not  yet  have  the  universal  algorithm,  we  must  continue  to  develop  new 
programs  and  upgrade  our  old  ones  as  the  nature  of  our  problems  change.  The 
challenge  is  to  get  the  cost  of  software  back  into  balance  with  total  systems  cost. 

Life  Cycle  Cost 

To  put  the  problem  in  perspective,  DOD  is  putting  emphasis  on  understanding 
the  total  cost  of  a system  over  its  life  cycle,  which  allows  the  proper  weight  to  be 
placed  on  cost  of  acquisition  in  the  total  cost  equation.  Many  studies  have  been  done 
which  show  that  quality  and  reliability  are  major  factors  in  the  life  cycle  cost  of 
software  systems. 

Dr.  David  S.  Alberts,  in  a Mitre  Corporation  report,  states  that,  conservatively. 
50  percent  of  total  life  cycle  cost  is  attributable  to  defects.  Moreover,  of  the  error 
types  studied,  he  reports,  "Design  errors  were  found  to  contribute  just  over  80 
percent  of  the  total  cost  of  error. ”J  This  represents  a significant  savings  opportunity 
as  well  as  a technical  challenge.  Perhaps  even  more  significant  than  the  potential  for 
saving  a large  portion  of  the  money  invested  in  software  is  the  possibility  that,  as  is 
the  case  with  oil,  we  may  not  be  able  to  produce  the  software  we  need.  Given  that 
one-half  of  our  programming  cost  is  attributable  to  defects,  then  we  may  postulate 
that  one-half  of  the  programming  population  is  not  productively  employed  because 
they  are  engaged  in  finding  and  fixing  errors.  This  means  that  the  demand  for 
programmers  must  grow  at  twice  the  demand  rate  for  new  program  applications. 
Even  assuming  the  demand  for  new  program  applications  is  constant,  the  program- 
mer population  will  have  to  double  each  year  unless  we  solve  the  productivity 
problem.  We  have  already  seen  that  quality  is  as  much  as  half  the  problem;  therefore, 
an  effective  QA  program  may  provide  as  much  as  half  the  solution. 

Failure  Mechanisms 

The  effective  QA  program  is  one  that  is  developed  based  on  an  understanding  of 
software  failures  and  the  errors  or  defects  that  cause  them 
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The  mechanisms  for  software  failure  may  be  grouped  into  four  basic  error 
categories-  functional  specification,  logic  specification,  code,  and  documentation. 
This  grouping  is  convenient  since  it  represents  both  the  activity  and  the  end  product 
where  an  error  may  first  appear.  In  Figure  1 we  see  an  overview  of  some  error 
mechanisms  and  their  probable  causes.  We  see  that  our  natural  difficulty  in  dealing 
with  complex,  abstract  entities  has  been  exacerbated  by  an  imprecise  technique  in 
developing  functional  and  logic  specifications.  Furthermore,  insufficient  attention 
has  been  given  to  what  effect  environmental  conditions  will  have  on  the  operability 
of  the  system  after  it  is  delivered.  The  development  laboratory  is  a sterile,  clinical 
environment.  The  real  world  generates  data  and  conditions  that  are  seldom  consid- 
ered when  the  functional  specification  is  being  created.  Most  experienced  designers 
now  avoid  design  decisions  based  on  average  data  rates  and  do  provide  a safety 
margin;  however,  they  too  often  fail  to  include  a design  strategy  that  will  allow  the 
system  to  continue  to  function  when  the  instantaneous  data  rate  or  other  conditions 
exceed  the  design  margin.  Depending  on  the  mission,  users  may  find  this  situation 
intolerable. 

As  is  most  often  the  case,  if  the  cause  of  a problem  has  been  correctly  determined, 
the  solution  or  corrective  action  is  directly  suggested.  However,  before  corrective 
actions  are  taken  it  is  prudent  to  review  the  error  distribution  and  cost.  This  review 
will  allow  the  proper  priorities  to  be  placed  in  the  get-well  scheme. 


Figure  1 
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Error  Distribution 

If  we  look  at  the  error  distribution  in  a large  data  base  compiled  over  a number  of 
years  and  across  a number  of  programs,  we  see  that  60  percent  of  the  life  cycle 
defects  (that  is,  the  sum  of  all  defects  recorded  both  during  development  and  after 
delivery)  are  attributable  to  errors  in  the  functional  and  logic  specifications.  For  the 
purpose  of  discussion  these  may  be  grouped  together  and  referred  to  as  design  en  ors. 

Design  errors  are  the  major  contributor  to  the  quality/productivity  problem. 
They  represent  40  to  60  percent  or  more  of  the  total  defects  and  represent  as  much  as 
80  percent  of  the  total  error  cost  to  fix.  A primary  reason  for  the  high  cost  is  that  in 
the  old-style  programming  process,  design  errors  are  typically  discovered  late  in  the 
test  cycle,  when  they  require  more  effort  to  correct.  There  is  general  agreement  that 
the  cost  of  correcting  an  error  is  a function  of  the  age  of  the  error.  The  cost  to  correct 
a design  error  late  in  the  system  test  phase  is  at  least  10  and  as  much  as  100  times 
greater  than  the  cost  to  correct  the  same  error  while  still  in  the  design  phase  (Figure 
2).  The  age-to-cost  relationship  holds  true  for  all  error  types — the  later  the  find  the 
more  expensive  the  fix.  This  points  to  the  strategy  we  can  use  to  attack  the 
problem — the  strategy  of  prevention  and  early  detection  of  errors. 


Figure  2 
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Prevention  Techniques/Improved  Programming  Technologies 

Figure  3 shows  the  four  basic  error  categories  mentioned  earlier  with  the 
causative  factors  repeated.  The  prevention  techniques  listed  are  by  no  means 
exhaustive  or  detailed,  but  they  are  representative  of  a host  of  improved  program- 
ming technologies  that  are  being  used  with  increasing  success  today  While  quality 
programs  in  the  past  have  under-emphasi/ed  prevention  of  errors  as  a viable  process, 
the  notion  that  "getting  a program  right”  rather  than  “getting  it  to  work”  is  now  in 
vogue.  Dr.  Harlan  Mills.  IBM  Federal  Systems  Division  Director  of  Software 
Engineering  and  Technology,  in  his  paper  on  software  development,  says  that 
“...getting  programs  to  work  is  a by-product  of  getting  them  right  " He  further 
states: 

Since  it  is  well  known  that  no  foolproof  methods  exist  for  know  ing  that 
the  last  error  in  a program  has  been  found,  there  is  much  more  practical 
confidence  to  be  gained  in  never  finding  the  first  error  in  a program,  even  in 
debugging.  Ten  years  ago  this  would  have  been  dismissed  as  unreal  But  it 
is  happening  with  regularity  among  gcxxl  programmers  today 

The  reason  program  correctness  is  key  to  go<xl  program  design  is  that  a 
discipline  of  rigor  is  imposed  in  place  of  the  currently  widespread  heuris- 
tics. Structured  programming  is  marked  by  a stepwise  refinement  design 
process,  in  which  programs  are  derived  and  validated  as  successive  func- 
tion expansions  into  simpler,  more  primitive  functions.  At  first  glance, 
stepwise  refinement  may  simply  look  like  an  orderly,  top-down  sequence 
for  inventing  program  statements,  but  there  is  more  at  stake  in  going  from 
heuristic  invention  to  rigorous  derivation.  What  is  at  stake  is  a visible 
design  structure  that  survives  the  coding,  for  use  in  maintenance  and 
modification  as  well  as  implementation.  Each  refinement  marks  the  top  of 
a hierarchy  which  can  serve  later  as  a new  intermediate  starting  point  for 
verifying  correctness  or  adding  capability  to  a program.' 

The  improved  programming  technologies  have  been  implemented  in  the  IBM 
Federal  Systems  Division.  Mixed  results  have  occurred  in  some  cases  because  of  a 
large  variation  in  understanding  of  the  techniques  among  programmers  and  pro- 
gramming managers.  Programming  managers  are  naturally  hesitant  to  change  a 
process  or  technique  that  has  been  successful  in  the  past  However,  old  methods  are 
no  longer  appropriate  because  we  are  looking  at  success  in  a new  , more  quantifiable 
way.  The  new  measure  is  how  a program  will  perform  over  its  life  cycle,  not  just 
development  cost  and  schedule 

In  our  drive  to  reduce  overall  programming  cost,  we  must  build  the  attitude  to 
utilize  the  improved  programming  technologies  with  the  rigor  intended  This  is  the 
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Figure  3 

CORRECTIVE  ACTION:  ERROR  PREVENTION 
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only  way  we  will  achieve  the  full  potential  offered.  However,  it  is  not  all  attitude.  The 
Federal  Systems  Division  has  recognized  that  consistent,  efficient  implementation  of 
i pi  proved  programming  technology  must  be  based  on  a common  understanding  of. 
and  approach  to,  these  new  methods.  Consequently,  a software  engineering  educa- 
tion program  has  been  implemented  which  is  aimed  at  bringing  every  programmer 
and  programming  manager  up  to  a common  understanding  of,  appreciation  for,  and 
ability  to  apply  a rigorous  engineering  approach  to  designing  and  implementing 
software  systems.  This  program  goes  back  to  basics.  Starting  with  elementary  logical 
expression,  it  proceeds  through  systematic  programming  and  systematic  design 
courses  and  workshops.  Mills  says,  "We  build  programs  from  the  top  down  but 
should  learn  programming  from  the  bottom  up.”  Universities  have  provided  much 
of  the  research  for  the  software  engineering  discipline  but  generally  have  not  yet 
developed  undergraduate  curricula  to  specifically  deal  with  it. 

The  new  emphasis  on  QA  with  improved  programming  technologies  for  error 
prevention,  even  with  inconsistent  application,  has  shown  some  encouraging  results 
(Figure  4).  Twenty-eight  errors  per  thousand  source  lines  of  code  seems  very  high, 
but  programmers  have  been  traditionally  tight-lipped  about  how  many  errors  they 
find  and  fix  in  debugging  a program,  and  even  after  the  program  gets  into  test,  the 
actual  number  of  problems  has  not  always  been  reliably  recorded.  The  numbers 
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Figure  4 

ERROR  DISTRIBUTION  WITH  PREVENTION  MEASURES 
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shown  in  Figure  4 represent  an  honest  attempt  to  count  and  classify  all  errors  found 
during  both  the  development  and  operational  phases  of  the  program  life  cycle. 
However,  the  reduction  in  error  rate  from  55  to  28  errors  per  thousand  source  lines 
of  code  is  still  very  significant  and  we  should  begin  to  see  an  inflection  in  the  software 
cost  growth  curve  if  these  results  are  sustained  and  improved.  Obviously,  functional 
specification  errors,  which  are  in  general  the  most  costly  to  correct,  have  seen  the 
least  improvement.  The  lack  of  improvement  in  functional  specification  error  rate  is 
due,  perhaps,  to  the  fact  that  developing  functional  specifications  is  still  largely  an 
interpretive  process.  The  functional  specification  process  lacks  the  precision  we  have 
seen  developed  for  logic  design  through  the  use  of  the  graphic  rather  than  narrative 
technique.  This  is  an  area  where  additional  new  techniques  and  innovations  are 
required. 

Defect  Detection 

After  a full  measure  of  prevention  has  been  applied  to  the  development  process, 
we  must  still  deal  with  the  possibility  that  errors  have  been  made.  Errorless 
programming  is  still  an  academic  topic  rather  than  common  practice,  at  least  for  the 
present.  As  previously  shown,  the  earlier  a defect  is  detected  the  cheaper  it  is  to 
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correct.  Not  only  is  it  cheaper,  it  is  really  more  correctable.  How  often  have  we  seen  a 
less  than  optimum  fix  installed  because  there  isn't  money,  time,  or  ability  to  go  back 
and  reprogram  correctly?  There  are  still  many  systems  in  the  military  whose  target 
machines  are  very  efficient  for  the  application,  but  terribly  inefficient  as  a develop- 
ment base.  Sometimes  days,  or  even  weeks,  are  required  in  an  inefficient  environ- 
ment to  turn  around  new  source  code  and  do  a system  build.  As  a consequence,  there 
has  been  a high  reliance  on  object  patches  as  a means  of  getting  in  a fix  quickly. 
Unfortunately,  patches  sometimes  create  a bigger  problem  than  they  solve.  Patches 
seem  to  beget  patches,  and  eventually  continuity  or  correspondence  with  the  source 
is  lost.  Hence,  if  errors  are  found  immediately  after  coding  a module,  the  fix  is 
usually  not  difficult  and  only  the  original  compilation  is  lost.  Programmers  have 
always  felt  it  easier  to  debug  their  code  on  the  machine  using  test  cases  that  they 
themselves  have  generated,  but  new  evidence  shows  that  it  is  not  easier.  Testing  most 
often  only  shows  that  there  is  an  error,  or  unexpected  results.  It  does  not  pinpoint  the 
source  of  the  error.  The  test  may  provide  a clue,  but  the  final  analysis  and  discovery 
is  done  by  the  programmer  with  the  listing,  reading  the  code.  If  this  is  true,  then  why 
not  read  it  in  the  first  place?  Code  reading  has  often  been  practiced  by  programmers 
with  good  results.  We  now  understand,  however,  that  going  one  step  further  will 
yield  even  better  results.  The  code  should  not  only  be  read,  but  inspected.  This 
should  be  done  by  a team  that  has  been  given  a road  map  to  follow  and  some  time  to 
prepare.  A moderator  should  be  assigned  to  conduct  the  session,  and  accurate 
records  of  the  findings  must  be  kept.  The  programmer  should  follow  up  on  the 
corrections  made  to  the  code  before  it  is  allowed  to  enter  the  test  and  integration 
process. 

The  notion  of  axle  inspections  as  an  efficient  detection  process  has  been  put  forth 
by  Mike  Fagan,  who  has  written  extensively  on  the  subject  and  has  provided  good 
data  to  support  his  conclusions.*  Fagan  has  not  only  shown  that  the  inspection 
technique  is  good  for  axle  but  has  proposed  it  as  a design  evaluation/error  detection 
tool  as  well. 

Design  and  code  inspections  have  been  implemented  on  several  large  programs  in 
the  Federal  Systems  Division  of  IBM  and  the  results  have  been  found  to  compare 
quite  favorably  with  those  published  by  Fagan.  Figure  5 shows  some  results  on  three 
programs.  A learning  process  in  the  inspection  technique  is  evident  because  all  three 
programs  were  similar  in  size,  complexity,  language,  and  application.  The  program- 
mers also  had  a similar  level  of  expertise:  hence,  it  was  felt  that  the  axle  should  have 
been  of  similar  quality.  Yet.  on  the  first  attempt  the  inspection  team  found  more 
benign  error  types,  violations  of  rules,  standards  and  conventions  than  were  found  of 
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Figure  5 

CODE  INSPECTION  ERROR  OCCURRENCE 
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the  malignant  error  types  (for  example,  logic  errors).  Note  that  this  technique  was 
not  received  enthusiastically  since  some  programmers  are  extremely  reluctant  to 
make  public  their  code  before  they  are  sure  it  has  been  completely  debugged. 
However,  early  publication  of  the  code  is  exactly  what  we  wish  to  encourage.  When 
programmer  and  machine  time  are  considered,  unit  testing  or  machine  debugging  by 
the  development  progtammer  is  beginning  to  look  like  one  of  the  least  efficient 
removal  techniques  in  terms  of  the  number  of  defects  found.  The  effectiveness  of  unit 
test  is  very  difficult  to  measure  because  it  has  always  been  considered  the  program- 
mers' personal  domain,  and  they  do  not  wish  to  change  it  or  have  others  infringe 
upon  it.  We  really  do  not  have  enough  data  at  this  point  to  summarily  dismiss  unit 
testing  completely  in  lieu  of  inspection.  However,  most  of  the  changes  to  a program 
during  the  unit  test  phase  seem  to  involve  more  style  than  substance,  making  the 
program  better  rather  than  making  it  right  (at  least  in  the  eyes  of  the  programmer). 
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Defect  Detection  Process 

The  list  of  defect  detection  processes  commonly  used  in  many  Federal  Systems 
Division  programs  is  shown  in  Figure  6.  Design  and  code  inspections  are  the  newest 
additions  to  this  list  and  from  an  early  detection  point  of  view  seem  to  offer  the 
greatest  potential.  Traditional  test  programs  usually  find  only  about  half  of  the  total 
errors  created,  which  means  that  without  the  additional  reviews  and  inspections  the 
number  of  latent  errors  delivered  may  be  quite  high. 

A general  error  detection  profile  which  might  be  expected  with  the  implementa- 
tion of  the  reviews,  inspections,  and  tests  from  the  previous  list  is  shown  in  Figure  7. 
Obviously,  if  the  efficiency  of  inspections  can  be  increased,  the  resultant  increased 
slope  on  the  detection  curve  will  have  a significant  positive  impact  on  the  develop- 
ment schedule  and  life  cycle  cost.  If  most  of  the  errors  are  removed  before  the  test 
cycle,  the  cost  of  this  most  expensive  part  of  the  development  program  will  go  down 
significantly  since  most  of  the  test  time  and  effort  is  devoted  to  finding  and  fixing 
errors. 

The  effect  of  improved  programming  and  defect  removal  techniques  can  be  seen 
in  Figure  8.  The  potential  for  improvement  is  clearly  a function  of  how  rigorously 
these  methods  are  employed.  Users  of  these  techniques  are  already  seeing  the  benefit 
Continued  emphasis  on  more  rigorous  implementation  of  these  basic  methods  will 
take  us  well  along  toward  the  goal  of  zero  defects,  and  will  have  a positive  effect  on 
programmer  productivity.  These  will,  in  turn,  lead  to  lower  life  cycle  cost  for 
software  systems.  II 
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Inspections 


Error  Detection  Profile 


Figure  8 

MODERN  PROGRAMMING  PRACTICES  AFFECT  SYSTEM  QUALITY, 
RELIABILITY  AND  PRODUCTIVITY  
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Dr.  William  J.  Perry 

Nearly  thirty  years  ago,  the  North  Atlantic  Alliance  united  15  democratic 
nations  bound  by  a common  desire  to  oppose  aggression  and  provide  a future  of 
liberty  and  freedom  for  their  children.  History  records  this  as  a most  successful 
alliance — no  other  has  brought  together  so  many  independent  nations  for  so  long. 

The  theme  for  this  conference,  “New  Directions  for  NATO,”  is  apropos  to  the 
coming  fourth  decade  of  the  Alliance.  The  conference  has  highlighted  the  prospects 
for  future  success  of  the  Alliance  as  well  as  the  serious  obstacles  and  problems  which 
remain  to  blunt  its  effectiveness.  It  is  time  for  the  Alliance  to  move  in  new  directions 
if  we  are  to  sustain  liberty  and  freedom  into  a fourth  decade. 

I would  like  to  focus  on  three  areas  of  importance.  First,  1 will  issue  a general  call 
to  quarters  to  emphasize  my  personal  concern  for  the  serious  challenge  which  we,  as 
an  Alliance,  face.  Second,  I will  discuss  actions  for  NATO  to  respond  to  this 
challenge — actions  aimed  at  sustaining  the  success  of  the  North  Atlantic  Alliance 
into  a fourth  decade  and  beyond.  Finally,  I will  provide  a progress  report — a report 
describing  the  status  of  initiatives  underway  to  stimulate  new  directions  for  NATO. 

A Call  to  Quarters 

The  effectiveness  of  the  Alliance  over  the  past  thirty  years  can  be  attributed  to  a 
common  political  purpose  supported  by  military  and  economic  strength.  This 
combination  has  provided  a shield  to  deter  aggression,  allowing  the  development  of 
independent,  vigorous  economies  which  have  provided  a fuller  and  more  prosperous 
life  for  the  citizens  of  Alliance  nations. 

Under  this  shield  of  stk.ngth,  the  Alliance  has  been  able  to  "go  it  alone,"  placing 
the  economic  interests  of  each  independent  nation  above  the  interests  of  a strong  and 
effective  Alliance.  In  contrast,  the  Soviet  Union  and  the  Warsaw  Pact  have  focused 
not  on  independence  and  consumer  goods  for  their  citizens,  but  on  monolithic  power 
building.  The  Soviets  have  been  spearheading  this  effort,  having  increased  their 
defense  expenditures  at  a compounded  rate  of  3-4  percent  per  year  for  nearly  two 
decades.  They  have  overcome  a 10-to-l  inferiority  in  the  central  strategic  balance, 
having  now  reached  essential  equivalence. 

In  this  environment  of  essential  equivalence,  the  strength  of  our  conventional 
forces  becomes  more  important  But  the  strength  of  Alliance  conventional  forces  is 
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being  challenged  by  the  same  program  of  Soviet  power  building.  While  I believe  our 
deterrence  remains  viable,  the  trends  will  give  us  cause  for  concern  during  the  fourth 
decade  of  the  Alliance. 

The  conventional  military  strength  of  NATO  vis-  a -vis  the  Warsaw  Pact  may 
be  viewed  as  a moving  pendulum.  While  the  position  of  the  pendulum  today 
continues  to  reflect  the  fundamental  strength  of  the  Alliance,  the  momentum  of  the 
pendulum  has  been  in  the  direction  of  the  Warsaw  Pact  for  some  time,  and  it 
continues  in  that  direction. 

In  comparing  the  quantity  of  equipment  deployed  by  the  Alliance  with  that  of 
the  Warsaw  Pact,  we  have  become  accustomed  to  a 2-to-l  or  greater  numerical 
advantage  by  the  Warsaw  Pact  in  most  weapon  categories.  In  the  past,  much  of  the 
Warsaw  Pact  equipment  was  outdated  and  inferior  in  quality,  and  we  could  sustain 
deterrence  on  the  basis  of  the  clear  qualitative  superiority  and  diversity  of  our 
weapons.  Hut  the  sustained  program  of  Warsaw  Pact  power  building  has  begun  to 
erode  our  previous  qualitative  edge  in  many  types  of  equipment. 

Consider,  for  example,  Warsaw  Pact  ground^irces  opposite  the  NATO  Central 
Region.  About  three-fourths  of  the  older  T-54  and  T-55  medium  tanks  in  the  region 
have  been  replaced  by  more  modern  T-62  and  1-64  models.  Other  significant  new 
systems  include  the  T-72  tank,  the  innovative  BMP  infantry  combat  vehicle,  self- 
propelled  artillery,  and  a number  of  organic  air  defense  systems.  The  new  artillery 
includes  a mobile  multiple  rocket  launcher,  and  self-propelled  122mm  and  I52mm 
guns.  Organic  air  defenses  include  the  ZSU-2.V4  fully-tracked,  radar-assisted  anti- 
aircraft guns,  and  live  types  of  mobile  or  man-portable  surface-to-air  missiles. 

Much  of  this  new  equipment  is  comparable  to  or  better  than  equipment  deployed 
in  NATO  today.  Furthermore,  the  equipment  is  generally  compatible  and  interoper- 
able, since  standardization  between  nations  in  the  Warsaw  Pact  is  imposed  by  fiat.  In 
contrast,  NATO  ground  forces  depend  upon  armaments  that  cannot,  in  many  cases, 
be  used,  supplied,  or  maintained  by  Allied  forces  aside  from  the  country  that 
developed  them.  For  example,  a recent  General  Accounting  Oftice  report  noted  that 
of  208  items  used  in  a NATO  army  group,  very  few  are  common  to  all  four  national 
forces  in  that  group,  and  the  bulk  are  unique  to  individual  forces. 

Turning  to  tactical  air  forces  in  the  Central  Region,  NATO  is  again  at  a nearly  2- 
to-l  numerical  disadvantage.  But  the  qualitative  superiority  of  NATO  tactical  air 
forces,  superior  pilot  training  and  maintenance,  and  the  ability  to  rapidly  deploy 
U.S.  tactical  air  forces  to  Kurope  largely  redress  this  disparity.  Yet.  looking  to  the 
future,  there  is  cause  for  concern. 

The  Soviet  Union  is  modernizing  its  air  force  with  technologically  improved 
aircraft.  In  the  past,  Soviet  tactical  air  forces  have  been  dedicated  to  primarily 
defensive  roles.  But  modern  Soviet  air  forces,  equipped  with  MIG-2.ts  and  27s,  and 
SU-17s  and  1 4s,  have  substantially  improved  range  and  payload  capabilities.  The 
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result  is  a new  capability  for  deep  air  superiority  and  interdiction,  providing  the 
ability  to  attack  high-value  targets  such  as  command  centers,  stockpiles,  and  ports  in 
Western  Europe.  We  believe  that  the  avionics  and  electronic  warfare  capabilities  of 
these  aircraft  have  been  upgraded,  placing  them  on  a par  with  the  F-4. 

Again,  the  concern  is  with  momentum — as  the  Soviets  maintain  production 
advantages  that  are  typically  2-to-l  for  most  modern  equipment;  as  they  improve  the 
quality  of  their  equipment;  as  they  expand  the  diversity  of  equipment  and  deploy  it  to 
Warsaw  Pact  forces  which  are  standardized  and  interoperable,  the  deterrence  of  the 
Alliance  may  be  tested.  The  principal  challenge  in  the  fourth  decade  of  NATO  will 
be  to  offset  this  Soviet  power  building  and  restore  the  momentum  of  the  Alliance. 

This  challenge  is  a serious  one — but  one  which  we  can  meet  given  the  will  and 
determination  to  act  efficiently  on  an  Alliance  basis. 

Actions  in  Response  to  the  Challenge 

To  respond  to  this  challenge,  we  need  not  match  the  Warsaw  Pact  man-for-man 
or  gun-for-gun.  We  should,  instead,  rely  on  our  fundamental  strengths  and  exploit 
the  fundamental  weaknesses  of  the  Warsaw  Pact.  Some  of  our  fundamental  strengths 
are  displayed  by  comparing  NATO  and  Warsaw  Pact  assets.  NATO  has  nearly  a 3- 
to-1  advantage  in  gross  national  product  and  a 3-to-2  advantage  in  population.  These 
ratios  underscore  the  fundamental  strengths  of  the  aggregate  Alliance  economic, 
industrial  and  personnel  resource  bases.  I believe  the  following  four  actions  are 
necessary  to  exploit  these  strengths: 

/.  Increase  Defense  Expenditures  Three  Percent  Per  Year  in  Rea I Terms. 

Increased  expenditures  would  stop  the  continuing  erosion  in  real  Alliance 
defense  budgets  and  enable  the  Alliance  to  respond  to  the  steady  growth  in  Warsaw 
Pact  defense  expenditures  over  the  past  two  decades.  Looking  to  the  future,  three 
percent  per  year  may  or  may  not  be  enough  to  equal  the  future  increases  in  Warsaw 
Pact  expenditures.  We  don’t  know  what  those  increases  will  be.  While  a three 
percent  per  year  increase  in  expenditures  will  not  in  itself  redress  the  military 
balance,  it  is  a substantial  increase  for  democracies  to  make  in  light  of  competing 
requirements,  and  I believe  it  is  an  increase  that  is  likely  to  compensate  for  the 
adverse  trends  in  the  NATO-Warsaw  Pact  balance  if  these  expenditures  are  used 
efficiently  on  an  Alliance  basis. 

2.  Make  Better  Use  of  Defense  Expenditures  Through  Cooperative  Research 
and  Development. 

The  lack  of  effective  coordination  and  cooperation  within  the  Alliance  causes  the 
whole  of  our  defense  output  to  be  less  than  the  sum  of  the  contributing  national 
inputs.  Our  equipment  is  not  standardized  and  often  not  interoperable.  We  find 
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examples  of  limited  technological  quality  and  high  unit  costs  due  to  small  and 
inefficient  national  production  bases. 

Cooperative  research,  development  and  production  can  have  a major  impact  on 
both  the  economic  and  military  effectiveness  of  the  Alliance.  In  the  near  term, 
cooperation  can  improve  the  quantity  and  quality  of  equipment  by  minimizing 
redundant  R&D  and  providing  economies  of  scale  in  development  and  production 
In  the  long  term,  cooperative  R&D  will  improve  the  interoperability  of  our  forces  as 
standardized  equipment  is  developed  and  deployed  in  the  field.  Furthermore, 
standardized  equipment  will  return  additional  benefits  by  providing  economies  of 
scale  in  logistics  support  and  maintenance,  thereby  reducing  support  personnel  and 
the  cost  of  operations. 

The  advantages  of  cooperative  R&D  have  been  recognized  for  some  time,  but  the 
principal  stumbling  block  has  been  the  protection  of  individual  economic  interests. 
Previous  experience  with  standardization  has  led  our  European  allies  to  link 
cooperation  with  "buy  American."  I would  like  to  convince  each  of  you  here  today 
that  the  future  of  the  Alliance  rests  on  increased  cooperation  on  the  basis  of  a 
genuine  two-way  street. 

U.S.  industry,  and  to  some  extent  the  U.S.  Congress,  perceive  that  increased 
cooperation  with  NATO  means  selling  less  of  our  equipment  to  Europeans,  thereby 
losing  the  economic  benefit.  They  further  perceive  that  the  impetus  for  a two-way 
street  derives  from  the  Department  of  Defense. 

As  I see  it,  the  Department  of  Defense  is  simply  being  realistic  in  responding  to 
the  changing  European  attitude.  I don’t  believe  that  Europe  will  accept  10-to-l 
procurement  ratios  in  the  future.  In  fact,  an  Independent  European  Program  Group 
has  been  established  to  rectify  this  imbalance,  and  they  are  taking  action  to  do  so.  So 
given  that  the  balance  is  going  to  change,  our  actions  should  be  directed  to  bring 
about  change  in  a manner  that  improves  NATO  effectiveness. 

Constructive  solutions  to  this  problem  will  recognize  that  the  European  defense 
industry  is  capable  of  undertaking  growing  responsibilities  in  both  prime  and  sub- 
contract roles.  Europe  has  a legacy  of  expertise  in  the  science  and  technology  of 
defense  material.  But  the  market  within  an  individual  nation  has  generally  been 
insufficient  to  warrant  investment  in  facilities  and  research  on  a scale  compatible 
with  efficient  production  on  a trans-Atlantic  basis.  A successful  program  of 
cooperation  will  recognize  these  fundamental  characteristics  and  their  implications 
for  a true  two-way  street. 

3.  Improve  A pplication  of  the  A Hiance  Technology  Base 

The  3-to-l  Alliance  advantage  in  GNP  underscores  our  fundamental  advantage 
in  industrial  base  and  supporting  technology.  There  is  no  doubt  that  our  fundamen- 
tal industrial  technology  is  dramatically  superior  to  that  of  the  Warsaw  Pact — yet  we 
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find  very  adequate  technology  in  their  recently  deployed  equipment.  They  are 
evidently  putting  their  best  technology  in  military  equipment  and  making  good 
decisions  about  where  technology  is  important 

While  we  have  substantial  leads  in  components  of  technology,  we  have  not  been 
effective  in  translating  this  lead  into  deployed  equipment — and  we  are  seriously 
deficient  in  applying  our  technology  effectively  on  an  Alliance  basis.  We  are  simply 
not  organized  to  apply  NATO  industrial  technology  to  efficiently  support  NATO 
military  effectiveness.  To  stimulate  effective  application  of  our  aggregate  technology 
base,  we  need  to  improve  industrial  cooperation  and  technology  sharing.  We  need  to 
extend  technology  sharing  to  all  levels— from  the  most  basic  equipment  to  the  most 
sophisticated. 

In  an  effective  program,  the  United  States  will  bear  a special  responsibility  for 
leadership.  In  protecting  the  security  of  the  United  States,  we  have  developed 
procedures  to  protect  our  technology  base.  These  procedures  may  act  as  hurdles  to 
an  ally  who  wants  to  cooperate  with  us.  We  need  ways  of  either  eliminating  or  at 
least  reducing  the  height  of  the  hurdles,  so  that  we  can  improve  the  application  of 
our  mutual  technology  base  on  an  Alliance  basis. 

Status  of  Initiatives  to  Stimulate  “New  Directions" 

A foundation  for  new  directions  is  provided  by  the  Long  Term  Defense  Program 
adopted  by  the  NATO  Summit  in  May.  This  program  is  a beginning  that,  one  year 
ago,  I would  not  have  thought  possible  considering  the  comprehensiveness,  detail 
and  response  to  military  needs  reflected  in  the  plan.  As  a vital  underpinning  for  this 
plan,  most  NATO  nations  have  confirmed  their  intention  to  increase  real  defense 
expenditures  at  the  rate  of  three  percent  per  year  over  the  next  several  years. 

The  heads  of  government  have  also  endorsed  a program  of  greater  Alliance 
cooperation  to  increase  collective  efficiency.  This  program  contains  a variety  of 
initiatives  intended  to  broaden  the  basis  of  cooperation.  Our  intent  in  these  initiatives 
has  been  to  encourage  free  market  “pull"  for  greater  cooperation  as  opposed  to 
government  "push."  We  are  attempting  to  remove  many  of  the  burners  to  trans- 
Atlantic  industrial  relationships,  and  clear  the  way  for  several  alternative  ap- 
proaches to  effective  cooperation 

1 he  traditional  approach  has  been  the  development  of  a cooperative  program  for 
each  specific  project.  Examples  of  recent  progress  taking  this  approach  include  the 
following: 

Global  Positioning  System 

A Memorandum  of  Understanding  (MOU)  was  approved  at  the  Apnl  Council  of 
National  Armament  Directors  (CNAD).  This  MOU  provides  for  joint  participation 
in  the  NATO  Global  Positioning  System  (N  AVSTAR),  emphasizing  the  design  and 
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application  of  common  user  equipment.  Participants  include  Belgium.  Canada, 
Denmark,  France,  Germany,  Italy,  the  Netherlands,  Norway,  the  U.K.  and  the  U.S. 
The  Global  Positioning  System  offers  an  extremely  efficient  means  of  supporting 
NATO  forces  at  all  levels  with  greatly  improved  navigation  information. 

JP-233  Munition 

The  U.K.  and  the  U.S.  are  sharing  the  expense  of  developing  the  JP-233 
munition,  with  the  actual  development  work  taking  place  in  the  U.K.  In  May  of  this 
year,  the  U.K.  and  the  U.S.  committed  to  full  scale  development  of  this  munition, 
which  will  provide  an  important  capability  to  deny  the  use  of  Warsaw  Pact  aircraft 
through  airfield  attack  and  runway  cratering.  Other  nations  are  being  invited  to  join 
the  cooperative  program. 

XM-1  Tank  Gun 

Since  1973,  the  U.S.  Army  has  engaged  in  a cooperative  effort  with  the  U.K.  and 
Germany  to  seek  a common,  optimal  tank  gun  for  NATO  forces.  In  January  of  this 
year,  the  Secretary  of  the  Army  recommended  that  the  German  120mm  gun  system 
design  begin  U.S.  development  and  testing  to  adapt  it  for  the  XM-1  tank.  The 
objective  is  to  allow  the  first  line  tank  forces  of  the  U.S.  and  the  Federal  Republic  of 
Germany  to  use  common  ammunition. 

Modular  Forward  Looking  Infra  Red  Systems 

The  U.S.  and  Germany  have  completed  an  MOU  on  the  Modular  Forward 
Looking  Infra  Red  (MOD  FLIR)  which  became  effective  in  April.  The  near  term 
objective  is  to  maximize  common  use  of  MOD  FLIR  systems  and  modules  by  U.S. 
and  German  armed  forces.  In  the  long  term,  we  believe  this  program  will  impact  on 
FLIR  systems  in  all  NATO  forces,  providing  the  military  advantages  of  modem 
night  vision  devices,  the  economic  advantage  of  shared  development,  and  the 
operational  advantages  of  interoperability. 

Improved  Sidewinder  Missile 

In  October  of  1977,  Germany  and  the  U.S.  completed  an  MOU  to  co-produce  the 
Improved  Sidewinder  Missile  (AIM-9L).  Implementing  arrangements  are  nearing 
completion  and  the  way  has  been  cleared  to  permit  the  transfer  of  production  data 
from  Germany  to  Norway  and  the  U.K.  With  that  step,  we  will  be  on  our  way  to  the 
start  of  the  European  Sidewinder  Co-Production  Program. 

While  these  examples  illustrate  recent  progress,  working  out  a cooperative 
approach  for  each  specific  project  is  often  difficult  and  deliberate.  Consider,  for 
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example,  the  efforts  associated  with  the  F-16  and  AWACS.  While  this  alternative  is 
welcome  where  it  is  achievable,  1 believe  a “package"  or  “family  of  weapons" 
approach  offers  far  greater  opportunities  for  successful  cooperation. 

The  package  approach  is  based  upon  identifying  a weapon  or  weapons  family 
having  considerable  operational  flexibility  within  a broad  operational  concept.  The 
next  step  is  to  identify  countries  or  groups  of  countries  to  fund  and  develop  the 
individual  weapons  in  the  package  The  fundamental  idea  is  to  improve  t'e 
probability  of  cooperation  by  broadening  the  basis  for  cooperation — broadening  the 
weapons  considered  as  well  as  the  countries  interested  in  participating. 

This  package  concept  provides  a mechanism  compatible  with  a two-way  street, 
and  a mechanism  which  allows  the  Alliance  to  benefit  from  comparative  advantages 
based  upon  ptevious  experience  or  capital  investments. 

As  an  example,  consider  ship-fo-ship  missile  systems  for  which  France,  Ger- 
many, the  U.K.  and  the  U.S.  have  a common  interest.  A potential  approach  would 
be  to  develop  long  and  short  range  versions  of  the  missile.  After  noting  common 
requirements,  the  U.S.  might  undertake  development  of  the  long  range  version  and 
the  U K.  might  combine  with  France  and  Germany  to  develop  the  short  range 
version.  At  the  completion  of  development  we  would  make  our  developments 
mutually  available  for  licensed  production  or  two-way  purchase  to  support  efficient 
production  runs.  Key  to  this  approach  is  harmonizing  requirements  early  in  the 
acquisition  cycle.  We  believe  the  Periodic  Armaments  Planning  System  (PAPS) 
being  developed  under  the  CN  AD  will  help  in  this  respect. 

We  are  currently  exploring  packages  in  several  mission  areas,  including  air-to- 
ground  munitions,  air-to-air  missiles,  anti-tank  guided  weapons,  and  anti-surface 
ship  missiles.  We  believe  that  this  package  approach  can  provide  a new  and 
fundamental  thrust  to  bring  about  effective  and  efficient  cooperation  in  the  spirit  of  a 
true  two-way  street. 

Another  means  of  progressing  on  the  two-way  street  is  to  offset  NATO  purchases 
of  U.S.  military  equipment.  We  have  examined  purchase  of  administrative  vehicles 
for  use  by  our  forces  in  Europe,  and  determined  that  we  could  save  3-5  percent  in  life 
cycle  costs  by  purchasing  the  vehicles  in  Germany.  In  May  of  this  year,  we 
announced  the  initial  award  for  225  vehicles.  The  total  program  is  expected  to 
involve  more  than  10,000  vehicles  costing  more  than  $100  million. 

While  we  have  much  to  do,  there  has  also  been  progress  in  a framework  for 
technology  sharing.  In  the  relevant  policy  statement  by  Defense  Secretary  Brown,  the 
key  sentence  is,  "Defense  will  support  the  transfer  of  critical  technology  to  countnes 
with  which  the  U.S.  has  a major  security  interest,  where  such  transfer  can:  (1) 
strengthen  the  collective  security.  (2)  contribute  to  the  goal  of  weapons  standardiza- 
tion and  interoperability,  and  (3)  maximize  the  effective  return  on  collective  NATO 
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investment  in  R&D."  The  key  phrase  here  is  "return  on  collective  NATO  invest- 
ment,” not  just  the  return  on  U.S.  investment.  This  policy  has  been  applied  in  a 
number  of  key  transfer  decisions  made  in  the  last  six  months. 

The  new  policies  we  are  undertaking  are  not  politically  motivated.  Our  objective 
is  to  make  more  effective  use  of  NATO  defense  expenditures  so  we  can  increase 
NATO  military  effectiveness.  We  believe  that  effective  technological  cooperation 
can  best  be  accomplished  on  a company-to-company  basis.  We  are  considering  such 
programs  in  several  areas,  and  I wish  to  generally  encourage  increased  trans- 
Atlantic  technological  cooperation  in  the  defense  industries.  Any  proposal  for 
technology  sharing  will  have  to  pass  one  key  test — will  it  increase  the  military 
effectiveness  of  the  Alliance,  and  do  so  on  an  efficient  Alliance  basis? 

Summary 

While  these  actions  display  some  signs  of  progress,  there  is  much  more  to  be 
done.  I have  issued  a call  to  quarters  to  emphasize  my  personal  concern  for  the 
challenge  presented  by  the  sustained  program  of  Soviet  power  building. 

The  Alliance  has  responded  to  this  challenge  with  a Long  Term  Defense 
Program.  This  program  provides  the  means  to  increase  defense  expenditures  at  a rate 
which  could  roughly  balance  Alliance  defense  expenditures  with  those  of  the 
Warsaw  Pact. 

Within  this  framework  are  a number  of  approaches  to  increasing  the  effective- 
ness of  our  expenditures  by  cooperating  on  an  Alliance  basis.  The  innovative 
package  approach  provides  a mechanism  for  cooperation  on  a true  partnership  basis, 
allowing  the  Alliance  to  develop  standardized  equipment,  benefit  from  comparative 
advantages,  and  operate  on  a true  two-way  street. 

New  directions  in  Alliance  cooperation  will  not  be  achieved  on  a business-as- 
usual  basis.  There  is  no  way  we  can  overcome  the  obstacles  I described  by  proceeding 
in  the  future  as  we  have  in  the  past.  But  I think  it  can  be  achieved,  and  I am 
committed  to  giving  my  best  efforts  to  make  it  happen. 

President  Carter  and  Defense  Secretary  Brown  have  made  a personal  commit- 
ment to  improving  the  effectiveness  of  the  Alliance.  The  U.S.  Congress  will  be 
watching  the  response.  I call  on  each  of  you  to  move  in  the  new  direction  of  increased 
Alliance  cooperation — cooperation  that  is  essential  to  sustaining  this  vital  Alliance 
into  a coming  fourth  decade.  | 
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actual  number  of  problems  has  not  always  been  reliably  recorded.  The  number* 


Announcement  of 
8TH  ANNUAL  DOD  ACQUISITION 
RESEARCH  SYMPOSIUM  and 
CALL  FOR  PAPERS 

The  8th  Annual  DOD  Acquisition  Research  Symposium,  jointly  sponsored  by  the 
Defense  Systems  Management  College  (DSMC)  and  the  Federal  Acquisition  Insti- 
tute (FAI),  will  be  held  2-4  May  1979  at  the  Naval  War  College,  Newport,  Rhode 
Island. 

The  symposium  has  a twofold  purpose.  The  first  purpose  is  to  develop  candid,  open 
discussions  between  government  and  industry  regarding  major  policy  issues  of 
concern  to  those  doing  business  with  the  government.  Secondly,  the  symposium  will 
provide  a forum  for  the  disclosure  of  research  accomplished  in  the  acquisition 
management  field.  The  symposium  will  primarily  consist  of  workshops  on  the 
following  issues: 

• DOD  Sponsored: 

Acquisition  Decision  Process;  Length  and  Impact 
A- 109  Impact  on  New  Development  and  Leveling  Effect 
Multi  Year  Authorization  Need?  Possibility?  Impact? 

Four  Step  Procurement 
Realistic  Cost  Estimating 
Missionized  RFP 
Concurrency 
Design  to  Affordability 
Acquisition  Process  Credibility 

• FAI  Sponsored: 

Civil  Agency  Contracting  Officer  Role 
Socio-Economic  Programs  Impact 
Commercial  Product  Specifications 

CALL  FOR  PAPERS:  You  are  invited  to  participate  in  the  resolution  of  pertinent 
issues  by  providing  the  results  of  your  research,  personal  expertise  or  ideas  to  the 
symposium  in  the  form  of  a paper  dealing  with  one  (preferably),  or  more  of  the 
above  issues,  or  a closely  related  matter.  Papers  judged  to  add  substantially  to  the 
understanding  or  possible  resolution  of  an  issue  will  be  referred  to  the  symposium 
panelists  for  consideration  prior  to  the  workshop  session  at  the  symposium.  These 
papers  will  also  be  considered  for  publication  in  the  Defense  Systems  Management 
Review.  Special  recognition  for  the  most  meaningful  papers  will  be  given. 

Abstracts  (200-500  words)  are  required  and  should  be  sent  to  the  Defense  Systems 
Management  College.  ATTN:  Program  Chairman  (Lt  Col  Robert  Machen).  Fort 
Belvoir,  Virginia  22060,  not  later  than  19  January  1979.  Abstracts  will  be  reviewed 


for  appropriateness  and  each  researcher  will  be  notified  of  acceptance  or  non- 
acceptance  by  I February  1979.  Detailed  instructions  for  preparation  of  the  final  (20 
page  limit)  paper  will  be  provided  with  notification  of  acceptance. 

Final  papers  must  be  received  by  the  program  chairman  not  later  than  6 April  1979 
in  order  to  be  eligible  for  special  recognition  at  the  symposium.  Researchers  who 
submit  acceptable  papers  will  be  invited  to  attend  the  symposium  and  the  appropri- 
ate workshop  sessions. 

SUMMARY  SCHEDULE 

19  January  1979  - Abstract  receipt  deadline 

1 February  1979  - Acceptance  notification  deadline 
b April  1979  - Final  paper  receipt  deadline 

2 May  1979  - Symposium  starts 


; 


